Run ./holt --version. Expect: exactly holt 0.1.0.
fable-terminal
Derived from: commit d2848dd (dev), 2026-08-08, against the local checkout at
Flows
check each step and final checkFirst contact: version, help, and the out-of-the-box prompt
Proves the binary runs and its own --help matches its own commands — then exposes the real failure a fresh user hits following the README verbatim.
Persona / precondition: a terminal at the repo root; no ~/.holt/config.yaml yet (this is the state of any fresh install — the file doesn't exist until you create it).
Run ./holt --help. Expect: a Usage: holt [options] block naming -p, --print, --model,--output-format, --max-turns, --session-id, --resume, -h, --help, -v, --version,
and a Commands: block naming eval, ingest-calibre, export, import.
Run the exact command the top-level README's "Running a Prompt" section tells you to run:./holt -p "Summarize this repository" --max-turns 5. Expect: either a real summary, or — if
the model isn't available — a message that tells you what to do next (pull a model, or pass--model). Currently FAILS: it exits 2 with exactlyerror: ollama /api/chat failed: 404 {"error":"model 'qwen3-coder-next' not found"} and no
further guidance. Root cause: defaultModel: "qwen3-coder-next" (src/config/types.ts) is not
a model any fresh install has pulled, and the shipped README never says to pull it or configure
an override — that instruction only exists in an internal build-fleet doc
(think-like-fable/build-fable-terminal/README.md), not in the product's own README.md.
Recover: rerun with an explicit, locally-available model, e.g../holt -p "Summarize this repository" --max-turns 5 --model gpt-oss:20b. Expect: a [router]
line naming the model and backend, then real prose about the repository, exit 0.
Final check: ls -t ~/.holt/sessions/*.json | head -1 names a session file whose mtime is
from step 4, not step 3 — the failed run in step 3 did not silently retry or leave a phantom
success behind.
Cascade routing: cloud tiers fall back to local cleanly
Proves the router's fallback ladder degrades to the local Ollama tier instead of hanging or crashing when cloud credentials are missing or wrong.
Persona / precondition: Ollama running locally with a chat model pulled (gpt-oss:20b here).
With no cloud credentials set at all:
[sensitive content omitted]
word: pong" --max-turns 2 --model gpt-oss:20b. Expect: exactly one router line —[router] using gpt-oss:20b via ollama: tier 4 (local) available — then pong, exit 0`. No
cloud tier is attempted or logged; a missing key is filtered out silently rather than tried and
failed.
[sensitive content omitted]
a line [router] using gpt-oss:20b via openai: tier 3 (gpt-5.5) available; a line starting[router] parked gpt-oss:20b via openai until (a timestamp) : unavailable: openai-compat[router] using gpt-oss:20b via ollama: tier 4 (local) available
[sensitive content omitted]; then pong. Exit 0` — a bad
cloud credential degrades the turn, it doesn't fail it.
Final check: grep -c router_event ~/.holt/sessions/<the-session-id>.jsonl from step 2 returns3 or more — the fallback trail is in the ledger, not just the terminal.
Resume replays the ORIGINAL prompt, not the new one
Proves session continuity — and exposes a confirmed defect: --resume silently discards whatever new -p prompt you give it.
Persona / precondition: a fresh session: ./holt -p "reply with the single word: pong" --max-turns 2 --model gpt-oss:20b --session-id demo-resume.
Confirm the opening turn answered correctly. Expect: pong in stdout, exit 0.
Resume the session with a genuinely different question:./holt -p "what number did I just say" --max-turns 2 --resume demo-resume --model. Expect: the model is asked "what number did I just say" and — since no number was
gpt-oss:20b
ever said — answers accordingly (e.g. that none was given). Currently FAILS: the session
ledger records "type":"resumed","prompt":"reply with the single word: pong" — the OLD prompt,
byte-identical to turn 1 — and the model answers pong again, as if the new -p text was
never read. Root cause confirmed at src/headless/run.ts:115:prompt = sessionState.prompt || prompt; unconditionally prefers the session's stored prompt
over the new one whenever the old one is non-empty, which it always is.
Repeat step 2 two or three more times. Expect (documenting current behavior, not endorsing
it): most repeats answer pong again; at least once during this investigation a repeat instead
errored error: backend completed without visible output; try again; diagnostic:. This secondary symptom correlates with the same
~/.holt/diagnostics/<timestamp>.json
duplicate-prompt replay from step 2 but was not independently root-caused beyond that
correlation.
Final check: grep -c "what number" ~/.holt/sessions/demo-resume.json returns 0 — the new
prompt text never even reached the saved transcript. This is the sharpest proof of the defect.
Headless mode refuses what it can't ask about
Proves the "ask" permission mode degrades to a safe, visible refusal — not a hang and not a silent bypass — when there's no human present to answer.
Persona / precondition: a scratch working directory OUTSIDE this repo (so nothing here edits project code); no project-level .holt/settings.yaml yet; the global ~/.holt/settings.yaml default permission mode is ask.
From the scratch directory: holt -p "run the shell command: echo hello-from-holt" --max-turns. Expect: a tool-step line reading exactly
3 --model gpt-oss:20b→ echo hello-from-holt (error: headless mode has no interactive prompt for bash (echo (the reason is truncated to 80 characters by the UI itself — that
hello-from-holt); pre-al
truncation point is exact), followed by the model's own prose explaining it can't run the
command. Exit 0 (the turn succeeds by reporting the refusal; the CLI process itself doesn't
error).
In the same directory, create .holt/settings.yaml containing:
allow:
- "Bash(echo *)"
and rerun the same prompt. Expect: the command actually executes this time — the tool-step
line shows a real result, not a headless mode has no interactive prompt error.
Final check: cat .holt/settings.yaml still shows only the rule you hand-wrote in step 2 —
the agent did not append its own rules to unblock itself further.
The agent cannot edit its own guardrails (GA-SPINE-LOCK)
Proves a security-floor invariant: no permission mode, allowlist, or bypass state lets a Holt-driven agent rewrite the file that governs it or the suite that grades it.
Persona / precondition: repo root. Verified here via the project's own shipped test suite — real, current output from this exact checkout, not re-driven end to end against a live local model in this pass (a small local model doesn't reliably choose the one exact tool call needed to trigger the guard on free-form request; the guard itself is exercised directly by the suite).
Run bun test test/headless/permission-gate.test.ts test/permissions/governance.test.ts.
Expect: every test in both files reports (pass), including ones named literally a headless and
run cannot mutate .holt/settings.yaml through tools (GA-SPINE-LOCK)a headless run, each asserting the refusal reason
cannot mutate the eval suite through tools (GA-SPINE-LOCK)
contains GA-SPINE-LOCK. Verified 2026-08-08: both pass.
Open src/permissions/governance.ts and check the refusal template at the two call sites
(lines 89 and 251). Expect: both read exactlyGA-SPINE-LOCK: refusing to let the agent mutate <reason> (<path>).
Final check: grep -c "GA-SPINE-LOCK also blocks bash commands" returns
test/permissions/governance.test.ts1 — the guard covers the bash-command path as well
as the tool-call path, not just one route in.
No first write without a first read (BC-EMPIRICAL)
Proves the write tool won't blind-create or blind-overwrite a file the agent hasn't looked at yet this session.
Persona / precondition: a scratch working directory with no pre-existing probe.txt.
Run: holt -p "create a file named probe.txt containing the word hello, using your write tool". Expect: a tool-step line reading exactly
--max-turns 3 --model gpt-oss:20b→ write probe.txt (error: refusing to write probe.txt: it has not been read this session, and
(BC-EMPIRICAL) — , 0ms)probe.txt is NOT created. Exit 0 (the turn completes by
explaining the refusal).
ls probe.txt. Expect: No such file or directory.
Final check: touch probe.txt && echo seed > probe.txt, then ask the agent to read it first
and only then overwrite it. Expect: once the file has been read this session, the same write
request succeeds. NOT VERIFIED this pass — flagged as an expectation to confirm, since it follows
directly from the guard's own stated condition ("has not been read this session").
Interactive REPL and its slash-command surface
Proves the terminal chat mode a human actually types into, and a representative slice of its /-commands, match their own shipped smoke tests character-for-character.
Persona / precondition: a terminal at (or outside) the repo root; run bare ./holt --session-id <any-id> with no -p and no --resume to land in the interactive REPL.
Launch it. Expect: the exact line Ctrl+D or /exit to quit when input is piped (a real TTY
instead shows the full boxed banner with version/backend/model), followed by a holt> prompt.
Type /help. Expect: exactly commands: /help /clear /compact /model /memory /resume /allow.
/offline /env /time /hunt /plan /tools /subs /transcode /dance /people /remember /forget
/lesson /lessons /corpus /study /wishlist /eval /exit /quit
Type /model qwen3:1.7b. Expect: exactly model set to qwen3:1.7b. This does not call any
model or backend — it only relabels the model for the next turn.
Type /allow Bash(git *). Expect: exactly allowed Bash(git *).
Type /offline. Expect: exactly offline on (toggles; a second /offline returnsoffline off).
Type /resume does-not-exist. Expect: exactly resume: no session found for does-not-exist.
Type /eval. Expect: exactly eval command is not available until E6 lands — an honest
"not built yet" rather than a silent no-op.
Type /exit. Expect: exactly bye, and the process exits with code 0.
Final check: grep -c '"type":"slash"' ~/.holt/sessions/<session-id>.jsonl returns 7 — one
ledger entry per slash command above (excluding /exit, which is logged as completed) — the
REPL is journaled exactly like a headless run.
The checked-in eval suite (`holt eval`)
Proves the conformance gate the README's Quickstart leads with actually runs, offline, with no keys and no Ollama needed.
Persona / precondition: repo root; nothing else required to be running (per the README: "does not require provider API keys or a local Ollama server").
Run ./holt eval. Expect: the line eval PASS; a workDir: line pointing into a fresh temp
directory; a Conformance: block listing 17 PASS lines, one per scenario (EV-EMPIRICAL,EV-HONEST-LEDGER, EV-DECISIVE, EV-STEELMAN, EV-FIX-CLASS, EV-DELEGATE,EV-RECOVERABLE, EV-ORCHESTRATES, EV-CASCADE, EV-OFFLINE, EV-GOVERNED, EV-PERSONA,EV-PEOPLE, EV-FABLE-ROUTING, EV-FABLE-REPEAT, EV-GROUNDED-EXPLAIN,EV-CONSTITUTION-SURVIVES); then a BC scores: block where every behavior-class score readsN/N (no partial scores). Exit 0.
Run it a second time immediately after. Expect: the identical PASS/FAIL shape — this is a
deterministic, stub-backed suite, not a flaky live-model one.
Final check: the workDir: path printed in step 1 no longer exists once the run completes
(ls <that-path> reports it missing) — it's scratch, not left behind.
The merge gate is currently RED (`bun run gate`)
Proves whether the project's own landing gate — the one docs/FLEET.md and scripts/gate.sh call the thing that must say "GATE PASS" — is actually green on a fresh checkout.
Persona / precondition: repo root, bun install already run, this exact checkout (commit d2848dd, dev branch, with the untracked .claude/ and think-like-fable/ directories present on disk exactly as they are today).
Run bash scripts/gate.sh (same as the gate script in package.json). Expect, per the
script's own header comment: it prints GATE PASS and exits 0, or GATE FAIL:<checks> and
exits 1 naming exactly which checks failed. Currently FAILS to be green: it printsGATE FAIL: lint test and exits 1.
Look at the lint failure alone: bun run lint. Expect (documenting the current state): 3
formatting errors from biome, all in untracked kit-scaffolding files that aren't holt product
source — .claude/settings.local.json, think-like-fable/templates/report.schema.json, and a
third file holding model-pricing numbers. The gate's biome check . is not currently scoped
away from these paths.
Look at the test failure alone: bun test. Expect (documenting the current state): exactly 2
of 383 tests fail — live: ollama /api/chat streams text and a forced tool call and live:, both in
ollama /api/chat forces a real tool call and parses name + argumentstest/backends/ollama.test.ts — both because they hard-require a real local Ollama server with
the model qwen3-coder-next pulled (the same missing model from F1). The remaining 379 tests
plus 2 skipped pass cleanly (2367 expect() calls, 55 files, roughly 92s).
Pull the missing model and re-check: ollama pull qwen3-coder-next (not run in this pass — a
multi-gigabyte network pull is out of scope for a read-only verification without separate
sign-off), then bun test test/backends/ollama.test.ts. Expect: both live: tests pass once
the model is present locally. NOT VERIFIED this pass.
Final check: bun run typecheck and bun run build — the other two gate checks — both pass
cleanly on their own, confirming the gate's redness is scoped to lint plus the 2 live tests, not a
wider break.
Backup and restore a Holt home (export / import round trip)
Proves the Pi-migration path documented in MIGRATE.md actually carries state across a fresh ~/.holt.
Persona / precondition: run with an isolated HOLT_HOME env var pointed at scratch directories, so this flow never touches your real ~/.holt.
Create a session in a scratch home: HOLT_HOME=/tmp/holt-a ./holt -p "reply with the single. Expect:
word: pong" --max-turns 2 --model gpt-oss:20b --session-id backup-demopong, exit0.
Export it: HOLT_HOME=/tmp/holt-a ./holt export /tmp/holt-a-backup.tar.gz. Expect: exactly two
lines — exported /tmp/holt-a-backup.tar.gz then manifest host=<your-hostname> models=<n>.
Import into a second, ALREADY-EXISTING scratch home, without --force:mkdir -p /tmp/holt-b && HOLT_HOME=/tmp/holt-b ./holt import /tmp/holt-a-backup.tar.gz. Expect:
it refuses rather than overwriting — exactly refusing to replace existing /tmp/holt-b; rerun, exit
with --force2.
Now force it: HOLT_HOME=/tmp/holt-b ./holt import /tmp/holt-a-backup.tar.gz --force. Expect
three lines: imported /tmp/holt-b; a line starting backup /tmp/holt-b.bak- (proving the
pre-existing directory was preserved as a dated backup, not destroyed); thenmanifest holt=0.1.0 host=<your-hostname> models=<n>.
Final check: cat /tmp/holt-b/sessions/backup-demo.json shows the same prompt, model, and"finalText": "pong" created in step 1 — the round trip is byte-faithful, not just
file-count-faithful.
Not covered and why
- Raspberry Pi hardware validation (LED
/danceon real Pironman5 hardware, direct-play
transcode on real Pi CPU): DEPLOYMENT.md and MIGRATE.md both explicitly defer this ("live Pi
validation remains a hardware-side check"); needs the actual device.
./holt ingest-calibre: needs a real Calibre library mounted at a/media/...path; a
full-shelf ingest is a multi-GB, multi-minute operation, not something to run inside a read-only
verification pass without separate sign-off.
- Cloud tiers 1–3 actually answering (rather than just failing over): needs real, valid
[sensitive content omitted]
failover path with a deliberately-broken key.
/tools install,/subs fetch|apply,/transcode apply: these mutate the filesystem or
install packages. Their read-only siblings (/tools, /subs, /transcode help) are covered in
F7; the --approve write paths are not.
- The Pi migration runbook end to end (
scpthe archive and the cross-arch binary to a real
second machine, per MIGRATE.md): F10 proves the archive round-trips correctly on one machine;
the actual cross-machine copy and install is unverified here.
- Real-TTY REPL rendering (the boxed input frame, spinner, terminal resize handling): F7
exercises the piped/non-TTY code path, which is deterministic and scriptable; the real-terminal
box-drawing path needs a human at an actual TTY to judge visually.
- **
/hunt,/plan,/people,/remember,/forget,/lesson,/lessons,/corpus scan,
/study, /dance, /wishlist**: smoke-registered in src/tui/slash.ts and exercised by the
project's own tests, but not walked here — F7 covers a representative slice, not the full
registry, to keep this pass to a reasonable size.
Results
| Flow | Verdict | Failures → tickets |
|---|---|---|
| F1 first contact | ||
| F2 cascade routing | ||
| F3 resume | ||
| F4 headless permission gate | ||
| F5 GA-SPINE-LOCK | ||
| F6 write-before-read guard | ||
| F7 REPL + slash commands | ||
| F8 eval suite | ||
| F9 merge gate | ||
| F10 backup/restore |