.offrails/guides/dev-tools.md
Dev Tools (/dev, a small Phoenix LiveDashboard)
TL;DR
- Every app gets
/dev(index, routes, system, mailbox, mailer previews) and/admin/jobs, built bynew, never stubbed; mounted only in development (/admin/jobsneeds admin access in production), which listens on 127.0.0.1 and refuses foreignHostnames (no login on/dev).- Kernel pages have their own layout: every link into them carries
hx-boost="false", every response isCache-Control: no-store, and they use noconfirm()dialogs.- Pending migrations: in development every HTML request shows a page with a Run migrations button;
/upis 503 in production while any are pending.- Agents read email via
/dev/mailbox/latest.json?to=…; the kernel-page browser smoke is.offrails/tools/browser-smoke/run.sh.- Ask the running app, don't guess:
GET /dev/introspect.json(orapp introspect) answers routes, redacted config, migrations, jobs, streams and subscribers, assets and CSS staleness. Development only; a test proves it's absent in production.- HTTPS in development only with a locally trusted certificate (mkcert). Chrome drops
Set-Cookieon a self-signed one even after "Proceed (unsafe)", so login bounces. No mkcert:bin/devserves plain HTTP and says why. Smokes never pass--ignore-certificate-errors.- Hot reload:
bin/devrebuilds and restarts on Rust/template edits, Tailwind watches CSS, andstatic/dev-reload.jsreloads open tabs (a 1 s poll of/dev/reload, never a held stream).Read the rest when changing a kernel page, debugging pending migrations, setting up dev HTTPS, or running the browser smoke.
Use the part. The dev-dashboard (and dev-mailbox) parts are this guide built, tested on SQLite and Postgres and security-reviewed, with more on top: /dev/dashboard, a LiveDashboard: live charts, requests per route, trace waterfalls, SQL fingerprints with N+1 detection and EXPLAIN, a live log tail, a routes explorer and a schema browser. When the product needs this, add the part rather than writing it from this guide: cargo offrails add-part dev-dashboard dev-mailbox (dependencies come first; workflow). Keep its // SECURITY: markers and the tests its CHECKS.md cites. This guide stays the spec: hand-write from it only when no part fits, or the human declined the part.
In development the app serves a toolbox at /dev. It's for the human clicking around and for the agent curling around. Keep it modest: it shows what the app already knows, with no metrics pipeline and no profiler.
generators/new.md builds every page below in every app. None is optional, stubbed or "added later".
| Route | What | Spec |
|---|---|---|
GET /dev | Index: tiles with live numbers. Mailbox (unread / total), Jobs (executing / available / retryable / discarded), Mailer previews (count), Routes (count), System (pending migrations). Try it buttons (Send a test email, Enqueue a test job). The agent curl cheat sheet below | here |
POST /dev/jobs/test | Enqueue a harmless CleanupJob; redirect to /admin/jobs/list with "Enqueued test job N." | here |
POST /dev/migrations | The pending-migrations page's Run migrations button | below |
/dev/mailbox (+ Previews tab at /dev/mailbox/previews) | Every email the app sent, rendered; mailer previews in the same viewer; JSON for agents | mailers |
/admin/jobs | The jobs dashboard (also in production, behind admin access) | jobs-dashboard |
GET /dev/routes?q= | The ROUTES table: method, path, handler, access badge. One GET search box over all three columns | routing |
GET /dev/system | APP_ENV, database backend + redacted host/file, FRONTEND_MODE, queues, inline worker, SMTP_URL configured or not, crate version + git sha (APP_REVISION), uptime; migrations applied and pending (Migrator::get_applied_migrations / get_pending_migrations_read_only, sea-orm-migration 2.0: the read-only variant never creates seaql_migrations) | here |
GET /dev/introspect.json | The app's state as JSON, for agents | below |
GET /dev/reload | {"version": "<boot id>:<asset digest>"}, polled by static/dev-reload.js | below |
GET /dev/mailers | 301 → /dev/mailbox/previews (previews live in the mailbox now). Build it as (StatusCode::MOVED_PERMANENTLY, [(header::LOCATION, "/dev/mailbox/previews")]): axum's Redirect::permanent sends 308 | mailers |
Files
src/controllers/dev.rs router(): /dev, /dev/routes, /dev/system, /dev/sockets (with channels), /dev/introspect.json, /dev/reload, /dev/mailers (redirect), POST /dev/jobs/test, POST /dev/migrations
src/kernel/introspect.rs snapshot(state, frontend_mode, live), BOOT_ID, reload_version() (the full file is below)
src/kernel/css.rs class_lists, missing_classes (template classes with no rule in static/app.css), css_escape, has_selector
static/dev-reload.js the reload poll (htmx mode; TS modes use Vite's own reload: islands rebuild with --watch and the poll sees the new digest)
src/controllers/migrations.rs pending(), the development guard middleware, the Run button handler
src/kernel/mailbox.rs router(PREVIEWS): /dev/mailbox/…
src/kernel/ui.rs shared chrome (with the jobs dashboard), BOOTED_AT for uptime
templates/kernel/dev/ index.html, routes.html, system.html, pending.html, sockets.html + _sockets.html (polled every 2s)
templates/kernel/mailbox/ index.html, _list.html, _message.htmlcontrollers/dev.rs holds a pub const FRONTEND_MODE: &str matching the app's AGENTS.md header. Config gains revision: Option<String> (APP_REVISION).
Rules
-
Mounted only when
config.env == AppEnv::Development, inroutes.rs:if config.is_development() { app = app .merge(dev::router()) .merge(mailbox::router(crate::mailers::previews::PREVIEWS)); }In production the routes don't exist (404, not hidden). Every
/devroute is inROUTESwithAccess::Dev, andsecurity_test.rsrequests each one with a productionConfigand expects 404. -
Reachable from this machine only.
/devhas no login, so development binds127.0.0.1(APP_BIND) andkernel::hostsrefuses anyHostthat isn't localhost,APP_URL's or anAPP_ALLOWED_HOSTSentry (DNS rebinding would otherwise read the mailbox same-origin). Opening it from a phone or a dev container takesAPP_BIND=0.0.0.0plusAPP_ALLOWED_HOSTS, and exposes the mailbox to that network (security). -
Server-rendered Askama + vendored htmx +
static/kernel/admin.css+static/kernel/admin.jsin everyFRONTEND_MODE, sharing the jobs dashboard's header (tabs: Overview · Jobs · Cron · Dev · Mailbox · Sockets · Routes) and its flash banner. -
Every link from an app page into a kernel page carries
hx-boost="false": the "Dev tools" link, the auth screens' "Check your mailbox" hint (paths::dev_mailbox()), any nav item into/admin/jobs. The htmx-mode app layout has<body hx-boost="true">, and a boosted navigation swaps only<body>, so a kernel page (its own<head>and stylesheet) renders unstyled until a hard refresh. Verified in Chrome.kernel_web_test.rs::app_links_into_kernel_pages_are_not_boostedscans an app page and fails on any boosted/devor/adminlink. (Inertia/react apps: use a plain<a href>, never<Link>/router navigation, for kernel pages.) -
Never cached, never stale on Back: every
/dev,/dev/mailboxand/admin/jobsrouter ends with.layer(kernel::ui::no_store()), so all their responses (pages, fragments, JSON, downloads, redirects) areCache-Control: no-store. The kernel layout setshistoryCacheSize: 0, andadmin.jsreloads pages restored from the back/forward cache. URLs of things that no longer exist redirect with a flash instead of a bare 404 (deleted message → inbox, deleted job → job list). -
Never display secrets: the database shows backend and host (or file path) only, never credentials (
redact_db, unit-tested), andSMTP_URLshows "configured" or "not set (dev mailbox)". -
In development the app layout's footer (htmx) or
Layoutcomponent (inertia/spa) shows a small "Dev tools" link to/dev. Render it only when the server saysenv == development(Ctx.dev/ a shared prop). -
bin/devmigrates first, then prints the exact URL with its scheme,→ https://localhost:${PORT}(orhttp://on the fallback, below) and…/dev (mailbox, jobs, routes), and once more as✓ ready: <url> (HTTP/2|HTTP/1.1)when the port accepts connections, so it isn't buried under build output.
HTTPS in development
Live pages need HTTP/2, and browsers only speak HTTP/2 over TLS (realtime). But a certificate the browser doesn't trust breaks login: after the user clicks "Proceed to localhost (unsafe)", Chrome silently drops Set-Cookie. The server answers 303 with a valid cookie, the account is created, and the browser bounces back to the login page ("email doesn't work"). Verified by a peer in a real Kanban; the same flow over http://localhost works.
So: dev serves HTTPS only with a locally trusted certificate.
| Situation | bin/dev-cert | bin/dev serves |
|---|---|---|
mkcert installed and its CA installed (mkcert -install, once per machine) | writes storage/dev-certs/{cert,key}.pem, exit 0 | HTTPS, HTTP/2: → https://localhost:3000 |
| mkcert installed, CA not installed (mkcert prints "the local CA is not installed in the system trust store") | refuses, deletes the files, says mkcert -install, exit 3 | plain HTTP/1.1, with the message below |
| no mkcert | refuses (no self-signed fallback), says how to install mkcert, exit 3 | plain HTTP/1.1, with the message below |
DEV_CERT_SELF_SIGNED=1 | an openssl self-signed certificate, with a warning that browsers drop cookies on it | both, from one server: HTTPS + h2 on PORT (for curl -k and scripted h2 checks) and plain HTTP/1.1 on APP_DEV_HTTP_PORT (default PORT+1) for the browser. bin/dev prints both URLs and says to open the http one |
APP_TLS=off | not run | plain HTTP/1.1, no message |
The fallback message says: no locally trusted certificate, so plain HTTP/1.1 at http://localhost:PORT; login and everything work; live updates stall past about 6 tabs (HTTP/1.1 allows ~6 connections per origin); install mkcert (brew install mkcert · apt install mkcert libnss3-tools), run mkcert -install once, restart bin/dev; self-signed certificates are refused because Chrome drops login cookies on them. bin/dev runs bin/dev-cert on every start (mkcert is instant), so a stale 0.6.0 self-signed certificate in storage/dev-certs/ is replaced or refused, never reused. The scripts are in new and realtime.
The bin/dev script
htmx mode (islands add pnpm --dir web exec vite build --watch --mode development &; inertia adds Vite; spa serves the Rust app over http behind Vite's proxy). Copy it; verified in the reference app in all three certificate situations. The PID file and process-tree stop were verified on macOS with a mock tree started from a non-interactive shell (the caller survives, every descendant and the PID file are gone, a second start refuses); --watch=always --minify with Tailwind 4.3.3 (a rebuild after a template edit stays minified):
#!/usr/bin/env bash
# Everything for development in one terminal: server (rebuilt on change) + Tailwind watcher.
set -euo pipefail
cd "$(dirname "$0")/.."
port="${PORT:-3000}"
# One PID file per server: stop exactly this bin/dev with `kill "$(cat storage/pids/dev-$PORT.pid)"`.
pidfile="storage/pids/dev-$port.pid"
mkdir -p storage/pids
if [ -f "$pidfile" ] && kill -0 "$(cat "$pidfile")" 2>/dev/null; then
echo "bin/dev is already running on port $port (PID $(cat "$pidfile")). Stop it: kill \$(cat $pidfile)" >&2
exit 1
fi
if command -v lsof >/dev/null && lsof -nP -iTCP:"$port" -sTCP:LISTEN >/dev/null; then
echo "port $port is taken by another process (lsof -nP -iTCP:$port -sTCP:LISTEN); pick another: PORT=… bin/dev" >&2
exit 1
fi
echo $$ > "$pidfile"
# On exit (Ctrl-C, `kill <pid>`, an error) stop our own process tree, leaves first, and nothing else.
# Never `kill 0`: started from an agent's shell, bin/dev shares that shell's process group, and
# `kill 0` would kill the caller too.
children() { ps -A -o pid= -o ppid= | awk -v p="$1" '$2 == p { print $1 }'; }
stop_tree() { local c; for c in $(children "$1"); do stop_tree "$c"; done; kill "$1" 2>/dev/null || true; }
cleanup() { trap - EXIT INT TERM; rm -f "$pidfile"; for c in $(children $$); do stop_tree "$c"; done; }
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
# HTTPS + HTTP/2 only with a locally trusted certificate (bin/dev-cert: mkcert + `mkcert -install`).
# Without one, plain HTTP/1.1: login works, live pages stall past ~6 tabs. APP_TLS=off forces http.
scheme=http
open_url="" # set when the URL a person should open differs from the main one
if [ "${APP_TLS:-on}" != "off" ]; then
if [ "${DEV_CERT_SELF_SIGNED:-}" = 1 ] && [ -f storage/dev-certs/cert.pem ] || bin/dev-cert; then
export APP_TLS_CERT=storage/dev-certs/cert.pem APP_TLS_KEY=storage/dev-certs/key.pem
scheme=https
if [ "${DEV_CERT_SELF_SIGNED:-}" = 1 ]; then
# Untrusted certificate: browsers drop cookies on it. Same server, plain HTTP beside it.
export APP_DEV_HTTP_PORT="${APP_DEV_HTTP_PORT:-$((port + 1))}"
open_url="http://localhost:$APP_DEV_HTTP_PORT"
fi
else
cat <<MSG
!
! No locally trusted certificate, so bin/dev serves plain HTTP/1.1 at http://localhost:$port
! Login and everything else work. Live updates stall past about 6 tabs (HTTP/1.1 allows ~6
! connections per origin; HTTP/2 needs HTTPS).
! For HTTPS + HTTP/2: install mkcert (brew install mkcert · apt install mkcert libnss3-tools),
! run 'mkcert -install' once, then restart bin/dev. Self-signed certificates are refused:
! Chrome drops login cookies on them even after "Proceed (unsafe)". APP_TLS=off hides this.
!
MSG
fi
fi
echo "Compiling and migrating (the first build takes several minutes)…"
cargo run -q -- db migrate
# Rebuild CSS on every template change, so a new class is never missing from static/app.css.
# --minify as in the production build: without it the watcher overwrites a minified file with an
# unminified one, and any check of the built CSS sees a different file in development.
bin/tailwindcss -i assets/app.css -o static/app.css --watch=always --minify &
# Migrate on every rebuild too, so a new migration never meets a stale database.
cargo watch -q -w src -w templates -w migration -x 'run -q -- db migrate' -x 'run -- serve' &
# The exact URL, scheme included (a bare localhost:$port is http:// to the browser), again once the
# server accepts connections, so it isn't buried under the build output.
if [ -n "$open_url" ]; then
echo "→ open in your browser: $open_url (HTTP/1.1; the certificate on :$port is self-signed and browsers drop cookies on it)"
echo "→ for scripts and h2 checks: $scheme://localhost:$port (curl -k; never a browser login)"
else
echo "→ $scheme://localhost:$port"
fi
echo "→ ${open_url:-$scheme://localhost:$port}/dev (mailbox, jobs, routes)"
(until (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; do sleep 1; done
echo "✓ ready: ${open_url:-$scheme://localhost:$port} ($([ "$scheme" = https ] && [ -z "$open_url" ] && echo "HTTP/2" || echo "HTTP/1.1"))") &
waitAPP_DEV_HTTP_PORT (development only; Config refuses it in any other environment, and it does nothing without TLS): serve binds a second, plain listener and serves the same router and AppState on it, so streams, presence and sessions are shared with the TLS port. The CSRF layer also trusts http://localhost:<port>. Dev session cookies aren't Secure, so the browser keeps them there. Verified by a peer (dev-92) in a stock Chrome profile and in the reference app: login sticks and the page goes live on the http port, while the self-signed TLS port fails the login smoke's preflight. bin/dev in that mode:
→ open in your browser: http://localhost:3001 (HTTP/1.1; the certificate on :3000 is self-signed and browsers drop cookies on it)
→ for scripts and h2 checks: https://localhost:3000 (curl -k; never a browser login)A bare localhost:3000 typed into the address bar is http:// to the browser. Against a TLS listener that used to show ERR_INVALID_HTTP_RESPONSE. serve's TLS branch now sniffs the first byte of each connection (a TLS handshake starts with 0x16) and answers anything else with a 307 to https://<Host><path> (kernel/tls.rs, deployment). 307, not 308: a cached permanent redirect would break APP_TLS=off on the same port later.
Never tell a user to click through the warning, and never run a browser check with --ignore-certificate-errors or acceptInsecureCerts: that's exactly the setting under which the bug is invisible. The framework's smokes launch Chrome with no certificate overrides and fail on an untrusted certificate (below).
Hot reload
Templates are compiled (Askama), so "hot reload" is the boring loop, all inside bin/dev:
- Rust and templates:
cargo watch -w src -w templates -w migrationrebuilds, migrates and restartsserve. - CSS:
bin/tailwindcss -i assets/app.css -o static/app.css --watch=always --minifyrewrites the CSS when a template orassets/app.csschanges. In development the asset catalog rescans per request, so the next page gets the new digest. - The browser: the layout loads
static/dev-reload.jswhenctx.dev. It pollsGET /dev/reloadonce a second while the tab is visible and callslocation.reload()whenversionchanges: a newBOOT_ID(the server restarted) or a newer file understatic/(versionisBOOT_ID:<newest mtime>:<file count>; mtimes, not digests, because every open tab asks once a second). While the server is down mid-rebuild, the fetch fails and it keeps polling.
Why a poll and not the signal stream or a socket: on the plain-HTTP fallback every held connection costs one of the ~6 per origin, and a reload stream in every tab would halve the tabs a live page survives. One short request a second costs nothing. Measured in the reference app: a CSS-only edit reached an open tab in 0.6 s; a template edit in about 17 s, of which ~9 s is the incremental build inside the watcher (Tailwind rebuilding alongside), ~1.7 s startup, up to 1 s the poll, the rest the page load.
Faster rebuilds: the skeleton Cargo.toml has
[profile.dev]
debug = "line-tables-only" # backtraces keep file:line; lldb shows no localsand .cargo/config.toml adds -C split-debuginfo=unpacked on Linux (macOS's default already; Windows MSVC refuses it, which is why it left [profile.dev] in 0.7.2). More build-speed settings, and what was measured: testing.
A peer measured a template edit going from 16–23 s to 3.6–5.9 s and a Rust edit from 11–12 s to 4.4–5.3 s (a Kanban, M2). In the smaller reference app on a newer machine incremental builds were ~5 s with or without it (not reproduced there; it costs nothing). For a debugging session with locals: CARGO_PROFILE_DEV_DEBUG=true cargo run. A page with unsaved form input loses it on reload; that's development.
// static/dev-reload.js: development only (the layout loads it when ctx.dev)
(() => {
let seen = null;
const tick = async () => {
if (document.visibilityState === "visible") {
try {
const r = await fetch("/dev/reload", { cache: "no-store", headers: { Accept: "application/json" } });
const v = r.ok && r.headers.get("content-type")?.includes("json") ? (await r.json()).version : null;
if (v && seen && v !== seen) return location.reload();
if (v) seen = v;
} catch {
// the server is restarting: keep polling
}
}
setTimeout(tick, 1000);
};
tick();
})();The layout line is {% if ctx.dev %}<script src="/static/dev-reload.js" defer></script>{% endif %}. /dev/reload and /dev/introspect.json pass the pending-migrations guard (like /up), so they answer while migrations are pending. TS modes: inertia/spa reload through Vite's own dev server or vite build --watch (islands), which changes the digest the poll sees.
Introspection
One way for an agent to ask the running app about its state, instead of grepping logs and guessing. GET /dev/introspect.json (development only: it's in dev::router(), so it doesn't exist in production or test) and app introspect (the same snapshot from the CLI; refuses unless APP_ENV=development; without the live parts, which only a serving process knows):
| Key | What |
|---|---|
app | name, version, frontend_mode, boot_id, booted_at |
url | app_url, tls, the browser protocol (h2, or http/1.1 and its ~6-tab limit) |
config | the Debug of Config: an allowlist, values only for never-secret settings and set/not set for the rest (config; never a raw env dump) |
routes | every ROUTES row: method, path, handler, access |
migrations | applied count, pending names, current |
jobs | counts by state and by queue (admin::stats); null while the table doesn't exist |
realtime (live) | open signal subscriptions, connections per member and transport, channel topics with subscribers and presence, frame counters, node(s). With per-stream pub/sub (realtime) it lists each stream and its subscriber count |
assets | the layout's digested asset URLs |
css | built, stale, and up to 50 missing_classes (template classes with no rule in static/app.css, from kernel::css::missing_classes) |
The whole file. Copy it, then change only the lines marked // APP:. It's the reference implementation an upgrading app wrote from this spec (axum 0.8.9, sea-orm 2.0.4, 2026-10-13), with the app-specific lines generalized:
//! Ask the running app, don't guess (.offrails/guides/dev-tools.md#introspection): the snapshot behind
//! `GET /dev/introspect.json` and `app introspect`, and the version `GET /dev/reload` hands to
//! `static/dev-reload.js`. Development only: the routes live in `controllers::dev::router()`, which
//! production and test never mount, and `app introspect` refuses outside development.
use std::fs;
use std::path::Path;
use std::sync::LazyLock;
use std::time::SystemTime;
use migration::{MigrationStatus, Migrator, MigratorTrait}; // migration/src/lib.rs: `pub use sea_orm_migration::MigrationStatus;`
use serde_json::{Map, Value, json};
use crate::kernel::ui::BOOTED_AT;
use crate::kernel::{css, jobs::admin};
use crate::routes::ROUTES;
use crate::state::AppState;
/// One id per process: a restart changes it, so open tabs reload.
pub static BOOT_ID: LazyLock<String> =
LazyLock::new(|| format!("{}-{}", std::process::id(), BOOTED_AT.timestamp_micros()));
/// The most missing classes listed (the count is always exact).
const MAX_MISSING: usize = 50;
/// `BOOT_ID:<newest mtime under static/ in ms>:<file count>`. Mtimes, not digests: every open tab
/// asks once a second.
pub fn reload_version() -> String {
let (mut newest, mut count) = (0u128, 0usize);
walk(Path::new("static"), &mut newest, &mut count);
format!("{}:{newest}:{count}", *BOOT_ID)
}
fn walk(dir: &Path, newest: &mut u128, count: &mut usize) {
let Ok(entries) = fs::read_dir(dir) else { return };
for entry in entries.flatten() {
let path = entry.path();
if path.is_dir() {
walk(&path, newest, count);
} else if let Ok(modified) = entry.metadata().and_then(|m| m.modified()) {
*count += 1;
let ms = modified.duration_since(SystemTime::UNIX_EPOCH).map(|d| d.as_millis()).unwrap_or(0);
*newest = (*newest).max(ms);
}
}
}
/// The app's state as JSON. `live` (the HTTP handler) adds what only a serving process knows.
pub async fn snapshot(state: &AppState, frontend_mode: &str, live: bool) -> anyhow::Result<Value> {
let c = &state.config;
let tls = c.tls.is_some();
let migrations = Migrator::get_migration_with_status(&state.db).await?;
let applied = migrations.iter().filter(|m| matches!(m.status(), MigrationStatus::Applied)).count();
let pending: Vec<String> = migrations
.iter()
.filter(|m| matches!(m.status(), MigrationStatus::Pending))
.map(|m| m.name().to_owned())
.collect();
// null while the jobs table doesn't exist yet (a fresh database before `db migrate`).
let jobs = match admin::stats(&state.db).await {
Ok(s) => json!({
"by_state": s.by_state.iter().map(|(st, n)| (state_name(st), json!(n))).collect::<Map<_, _>>(),
"by_queue": s.by_queue.iter().map(|(q, counts)| {
(q.clone(), Value::Object(counts.iter().map(|(st, n)| (state_name(st), json!(n))).collect()))
}).collect::<Map<_, _>>(),
}),
Err(_) => Value::Null,
};
let css = match css::missing_classes() {
None => json!({ "built": false, "stale": false, "missing_count": 0, "missing_classes": [] }),
Some(missing) => json!({
"built": true,
"stale": !missing.is_empty(),
"missing_count": missing.len(),
"missing_classes": missing.iter().take(MAX_MISSING).collect::<Vec<_>>(),
}),
};
let assets = state.assets.assets();
Ok(json!({
"app": {
"name": crate::config::APP_NAME,
"version": env!("CARGO_PKG_VERSION"),
"framework": fs::read_to_string(".offrails/VERSION").ok().and_then(|v| v.lines().next().map(str::to_owned)),
"frontend_mode": frontend_mode,
"env": format!("{:?}", c.env).to_lowercase(),
"boot_id": *BOOT_ID,
"booted_at": BOOTED_AT.to_rfc3339(),
},
"url": {
"app_url": c.app_url,
"tls": tls,
"dev_http_port": c.dev_http_port,
"protocol": if tls { "h2" } else { "http/1.1" },
"note": if tls {
"HTTPS with ALPN h2 (a locally trusted certificate)"
} else {
"plain HTTP/1.1: browsers hold ~6 connections per origin, so live pages stall past ~6 tabs"
},
},
"config": format!("{c:?}"), // Config's hand-written Debug redacts credentials (config.md)
"routes": ROUTES.iter().map(|(method, path, handler, access)| json!({
"method": method, "path": path, "handler": handler, "access": format!("{access:?}"),
})).collect::<Vec<_>>(),
"migrations": { "applied": applied, "pending": pending, "current": pending.is_empty() },
"jobs": jobs,
"realtime": if live { realtime(state) } else { Value::Null },
"live": live,
"assets": { "css": assets.css, "app_js": assets.app_js, "htmx": assets.htmx, "htmx_sse": assets.htmx_sse },
"css": css,
}))
}
fn state_name(st: &impl std::fmt::Debug) -> String {
format!("{st:?}").to_lowercase()
}
/// APP: what only the serving process knows about live connections. No realtime: `Value::Null`.
fn realtime(state: &AppState) -> Value {
json!({
// pubsub.streams(): (stream, receiver count) per live per-stream sender (realtime.md)
"streams": state.pubsub.streams().into_iter().map(|(s, n)| json!({ "stream": s, "subscribers": n })).collect::<Vec<_>>(),
"topic_count": state.pubsub.topic_count(),
// with channels: the /dev/sockets snapshot (connections per member and transport, topics, presence,
// frame counters, nodes). `Snapshot` derives Serialize.
// "channels": serde_json::to_value(state.hub.snapshot()).unwrap_or(Value::Null),
// with SSE presence: the label per room
// "presence": state.presence.labels(),
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reload_version_starts_with_the_boot_id() {
let v = reload_version();
assert!(v.starts_with(&format!("{}:", *BOOT_ID)), "{v}");
assert_eq!(v.split(':').count(), 3, "{v}");
}
}The handlers (controllers/dev.rs, both in router() and in ROUTES with Access::Dev) and the CLI arm:
/// `GET /dev/introspect.json`: the app's state for agents.
pub async fn introspect(State(s): State<AppState>) -> Result<Response, AppError> {
Ok(Json(kernel::introspect::snapshot(&s, FRONTEND_MODE, true).await?).into_response())
}
/// `GET /dev/reload`: polled once a second by `static/dev-reload.js`.
pub async fn reload() -> Response {
Json(serde_json::json!({ "version": kernel::introspect::reload_version() })).into_response()
}
// main.rs, `Command::Introspect` (needs the database, not the listener):
anyhow::ensure!(config.is_development(), "app introspect is development only (APP_ENV is {:?})", config.env);
let state = AppState::new(config, db)?;
println!("{}", serde_json::to_string_pretty(&kernel::introspect::snapshot(&state, controllers::dev::FRONTEND_MODE, false).await?)?);The router's .layer(kernel::ui::no_store()) already covers both responses. Apps without the jobs kernel drop jobs (and its import); api mode has no css or assets (drop both keys and the css import). Add pub fn streams(&self) -> Vec<(String, usize)> to PubSub (under the map lock: each stream and its receiver_count()) when the app has signals.
Tests (kernel_web_test.rs): in development it answers JSON with every key above, lists itself in routes, and contains none of a configured admin password or SMTP credentials; with a production and a test Config, /dev/introspect.json and /dev/reload are 404 and pages don't load dev-reload.js. Add both paths to ROUTES with Access::Dev.
The development CSS banner uses the same check. Every UI app has one: if Ctx has no css_warning field yet, add it (Option<String>, set by the Ctx extractor only when config.is_development(), rendered as the first plain-text line in <main>; frontend-htmx). An mtime check can't do this: Tailwind 4 doesn't rewrite unchanged output.
// src/kernel/css.rs (next to missing_classes)
/// The development CSS banner: the CSS is missing, or stale (template classes with no rule).
pub fn css_warning() -> Option<String> {
if let Some(problem) = crate::kernel::assets::app_css_problem() {
return Some(problem);
}
let missing = missing_classes()?;
let first = missing.first()?;
Some(format!(
"{CSS_BUILT} is stale: {} template class(es) have no CSS rule (first: {first}). bin/dev rebuilds it.",
missing.len()
))
}
// the Ctx extractor:
css_warning: if s.config.is_development() { crate::kernel::css::css_warning() } else { None },No confirm() dialogs in kernel pages
Kernel pages aren't boosted, so hx-confirm never fires there, and a native confirm() dialog would block the browser smoke tool. Destructive kernel actions are either cheap and dev-only (mailbox Clear all, deleting a job: no prompt), or go through a server-rendered confirmation page ("Retry 1,234 jobs?", which posts back with confirmed=1). If an app page needs a quick confirm, use data-confirm="…" on the form, handled in static/app.js (submit listener calling confirm()), and make browser tests accept dialogs (page.on('dialog', d => d.accept())).
Pending migrations
Like Rails' pending-migration error page and Phoenix's "There are pending migrations" screen: a database behind the code shows a clear page, not a "no such table" 500.
src/controllers/migrations.rs (generated by new):
/// Read-only: never creates seaql_migrations (a fresh database reports every migration pending).
pub async fn pending(db: &DatabaseConnection) -> Result<Vec<String>, DbErr> {
Ok(Migrator::get_pending_migrations_read_only(db).await?.iter().map(|m| m.name().to_owned()).collect())
}
pub async fn guard(State(s): State<AppState>, OriginalUri(uri): OriginalUri, req: Request, next: Next) -> Response;
pub async fn run(State(s): State<AppState>, Form(f): Form<RunForm>) -> Result<Response, AppError>; // POST /dev/migrations| Environment | Behavior |
|---|---|
| development | routes.rs adds app.layer(from_fn_with_state(state.clone(), migrations::guard)) after the routes and the .fallback(...) are added (a layer only wraps what exists when it's applied, so a fallback added later would skip the guard and unknown paths would 404 instead of showing the page). Every request except /static/*, /up and POST /dev/migrations gets a 503 "N pending migrations" page (kernel layout, so it works before any app table exists) listing them, with a Run migrations button: a CSRF-protected form posting return_to=<current path> to /dev/migrations, which runs Migrator::up, flashes "Ran N migrations." and redirects back (only to same-site paths) |
| production | Never auto-migrates: the entrypoint runs app db migrate before serve (deployment). The guard and /dev/migrations aren't mounted. GET /up returns 503 "pending migrations" while any is pending, so a deploy that skipped migrating never goes healthy |
worker (app worker, inline worker) | Before starting, checks pending(). If any are pending it logs once pending migrations: run \app db migrate`; worker waiting, rechecks every 5s (stopping cleanly on shutdown), then logs migrations applied; worker starting`. It never spams errors against missing tables |
Nothing in serve may read an app table before the listener is up. The guard renders from the kernel layout and seaql_migrations alone; everything else that touches app tables at boot (realtime's pub/sub and cluster, the inline worker, a cache warm-up) waits for pending() to be empty (realtime). Fresh-database smoke, after any change to serve or to what starts at boot (bin/dev migrates first, so it never shows this):
mkdir -p storage/pids && rm -f storage/fresh.sqlite3
PORT=4321 DATABASE_URL='sqlite://storage/fresh.sqlite3?mode=rwc' sh -c 'echo $$ > storage/pids/serve-4321.pid; exec cargo run -q -- serve' & sleep 5
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:4321/ # 503: the pending-migrations page, not a crash
curl -s -o /dev/null -w '%{http_code}\n' -X POST -H 'Sec-Fetch-Site: same-origin' -d return_to=/ http://localhost:4321/dev/migrations # 303
sleep 6; curl -s -o /dev/null -w '%{http_code}\n' http://localhost:4321/up # 200; the log says "migrations applied; realtime starting"
kill "$(cat storage/pids/serve-4321.pid)"; rm -f storage/pids/serve-4321.pid storage/fresh.sqlite3*bin/dev runs cargo run -- db migrate before serving, and again on every rebuild (cargo watch -x 'run -q -- db migrate' -x 'run -- serve'), so the page is mostly seen after a git pull with the server stopped, or after a rollback. Tests (kernel_web_test.rs): a fully rolled-back dev DB shows the page (with the migration names, the button and return_to), /up is 503, the kernel stylesheet still loads, the button applies all and redirects back with the flash, no open redirect; production with one pending: /up 503, / unaffected, /dev/migrations 404.
Browser smoke
HTTP tests can't see boosted navigation, <head> swaps, the back/forward cache or CSP violations. After changing kernel pages (or the app layout's links into them), drive a real browser:
-
Preferred: the chrome-devtools MCP (navigate, click, screenshot,
list_console_messages), in an isolated context. -
When it can't attach (Chrome's profile lock, CI, a script): the framework ships a ready smoke that launches the installed Chrome headless with a throwaway profile:
BASE=http://localhost:3000 bash .offrails/tools/browser-smoke/run.sh # Linux: CHROME=/usr/bin/google-chrome BASE=… bash .offrails/tools/browser-smoke/run.shIt opens
/devdirectly (styled?; an app's home need not link to it) → seeds 3 mails with the Send a test email button (each lands on its message,/dev/mailbox/<id>, with the flash "Sent a test email to your mailbox."; the list then shows at least 3) → opens one (styled, sandboxed iframe) → Clear all → Back (none of the cleared messages comes back) → Forward (judged by the cleared messages' ids, not an empty list: other smokesverify --fullruns alongside sign in, and an auth part's "New sign-in" email may arrive meanwhile) → (when/devhas an Enqueue a test job button) opens and deletes the job → Back (redirected to the list, flash shown), and fails on any CSP violation. Every step waits withwaitForNavigation/waitForSelector, with no fixed sleeps, and seeds only through the dev-only buttons, so it needs no app routes. -
Multi-tab live updates (any app with live pages): open N tabs in one profile on a live page, change something once, and assert every tab updates. It catches the HTTP/1.1 connection pool and an undersized per-user cap (realtime). The app supplies a small scenario module,
tests/e2e/live-tabs.scenario.mjs:export const path(the live page), optionallogin(page, base),mutate(page, base)returning a token, andseen(page, token)returning whether a tab shows it (the full contract is the header of.offrails/tools/browser-smoke/smoke-live-tabs.mjs):SCENARIO=tests/e2e/live-tabs.scenario.mjs BASE=https://localhost:3000 TABS=12 \ bash .offrails/tools/browser-smoke/run.sh live-tabsIt exits 1 if any tab misses the update, and says whether the page was on HTTP/1.1. Run it with 12 and 40 tabs after any realtime change.
-
Login sticks (every app with accounts; part of the UI check in the Definition of Done): a real browser with no certificate overrides fills and submits the real login (or sign-up) form, then proves a fresh page load is still logged in. It catches the dev-TLS trap above and any cookie a browser refuses (
Secureon http,__Host-with aDomain). The scenario exportslogin(page, base)andloggedIn(page):SCENARIO=tests/e2e/smoke.scenario.mjs BASE=<the URL bin/dev printed> bash .offrails/tools/browser-smoke/run.sh login -
axe in both color schemes: axe-core (WCAG 2.2 AA) on each path with
prefers-color-schemeemulated light, then dark; logged-out paths fromPATHSor the scenario'sa11yPaths, anda11yAuthPathsafterlogin. It finds abg-whitecard left light under a body that turned dark. The app's Playwright suite (test-standards §10) stays the CI gate; this is the framework's own check, and it needs no Node in the app:SCENARIO=tests/e2e/smoke.scenario.mjs BASE=… bash .offrails/tools/browser-smoke/run.sh a11y -
Every page after a boosted link (every htmx app): each GET page in
docs/routes.mdon a full load, then reached through a boosted link from/; the same buttons must react, with no console errors, and no page may load its own scripts in<head>. Signed out, then signed in as the demo account. No npm install (Node ≥ 18 and the installed Chrome);verify --fullruns it whenBASEis set (testing):BASE=<the URL bin/dev printed> node .offrails/tools/browser-smoke/smoke-boosted.mjs # from the app root
Every smoke launches Chrome without certificate overrides and starts with a preflight: an untrusted HTTPS certificate fails it with the explanation above, before any check runs. Verified in the reference app: against a forced self-signed certificate (DEV_CERT_SELF_SIGNED=1) login and live-tabs fail at the preflight; against the plain-HTTP fallback and against a trusted certificate both pass, and live-tabs reaches 12/12 tabs over h2.
Why it lives in .offrails/tools/ and not in the app: htmx-mode apps have no Node toolchain or package.json, and this check isn't part of the Definition of Done or CI. So run.sh installs puppeteer-core (pinned, verified 2026-10-12: 25.13.0, which needs Node ≥ 22.12; Node 24 LTS recommended) and axe-core (4.13.0) once into ~/.cache/offrails/browser-smoke and runs a copy of the script from there. The app's dependency tree, lockfiles and stack rules are untouched, and inertia/spa apps use the same script. It's optional tooling for agents and humans; the guaranteed checks remain the Rust tests above. Verified against the bookshelf app: 11/11 checks pass. Removing hx-boost="false" from the dev link, or omitting includeIndicatorStyles:false, makes it fail.
For agents
These endpoints are your eyes on the running app. Use them in the "ran the app and exercised it" step of the Definition of Done. Start with introspection; it answers most "why doesn't X work" questions before you read code:
curl -s $URL/dev/introspect.json | jq '.migrations, .css, .realtime, .url' # $URL: what bin/dev printed (-k never needed)
cargo run -q -- introspect | jq '.routes[] | select(.path | test("books"))'
curl -s "$URL/dev/mailbox/latest.json?to=ada@example.com" # did the email go out? follow .links[0]
cargo run -- jobs list --state discarded --json # did a job fail?
cargo run -- jobs stats # is anything stuck?
curl -s $URL/dev/system | grep -i pending # forgot to migrate?