Observify

User Tests · terminus3
← all user tests
USER TEST PLAN

terminus3

Derived from: no git repository (terminus3 is uncommitted); live filesystem + tool read on

0/40 checked · 0 pass · 0 fail

Flows

check each step and final check
F1

The structural gate holds across the whole corpus, not just the showcase unit

Proves Phase A of the shipped quality gate (_build/gate-task.sh) is a real floor under every delivered unit, in every pipeline stage — not just the one unit that has actually been accepted.

Persona / precondition: at ~/Projects/terminus3; no Docker needed (this flow only runs the GATE_FAST=1 structural phase).

0/4

Count task directories per stage: find tasks/working -mindepth 1 -maxdepth 1 -type d | wc -l
(and the same for tasks/reviewing, tasks/revising, tasks/accepted). Expect: 63,
3, 4, 1 — 71 task directories total.

Run GATE_FAST=1 _build/gate-task.sh <dir> against every one of those 71 directories and
tally results. Expect: all 71 runs end with the exact line GATE-TASK PASS, and zero
lines appear anywhere across the combined output — verified true on 2026-08-07 (71/71 pass,
0 failures).

Run it once more, alone, on the one unit that has actually cleared platform acceptance:
GATE_FAST=1 _build/gate-task.sh tasks/accepted/ball-valve-parametric. Expect: exactly
51 lines starting followed by GATE-TASK PASS.

Final check: grep the accepted unit's task.toml for category = "Hardware" and
subcategory = "CAD" — both present, matching the mission in company/CHARTER.md.

F2

The fleet's own ground-truth counter is broken

Proves (or disproves) Article II in the one place it matters most here: does the project's OWN declared "how much is done" command tell the truth? KNOWN DEFECT found while grounding this plan — write the step that exposes it.

Persona / precondition: at ~/Projects/terminus3; _build/project.env sourced.

0/4

Run source _build/project.env && eval "$ATOMS_DONE_CMD" — this is the literal ground-truth
command _build/progress.sh, _build/assess.sh, and the dashboard all consume. Expect (per
Article II, the real number of complete task dirs on disk):
a number at or near 71.
Currently fails: prints 0.

Run the shipped dashboard directly: bash _build/progress.sh. Expect: a fleet percentage
reflecting real completed atoms. Currently prints exactly fleet 0.0% (0/9 tasks)
confirmed via the shipped script itself, not a re-implementation.

Diagnose: ATOMS_DONE_CMD walks "$UNITS_ROOT"/*/ — one level below tasks/, i.e.
tasks/working/, tasks/reviewing/, tasks/revising/, tasks/accepted/ themselves — and
checks each of THOSE four directories for a top-level task.toml. None of the four stage
folders is itself a task, so the count is always 0. Real units live two levels down
(tasks/<stage>/<slug>/) since the corpus was reorganized into stage folders after
project.env was authored (comment on line 8 still says "each unit = one task dir
tasks/<task-name>/ directly"). Expect (corrected two-level walk): 71.

Final check: grep -n ATOMS_DONE_CMD _build/assess.sh — confirm assess.sh (the per-unit
audit tool) evaluates the same broken command, so this is not an isolated display bug: every
fleet-percentage or per-unit ground-truth read taken today through the shipped tooling
under-reports.

F3

Full landing gate (docker + oracle) passes end-to-end on every unit in review

Proves the units sitting in reviewing/ are technically finished, not technically broken — the docker builds, the verifier-isolation proof, and the oracle roundtrip all included.

Persona / precondition: Docker Desktop running; the fbtask-* images for these three units are already cached from prior gate runs (rebuilds will be fast, cache-hit).

0/4

Run _build/gate-task.sh tasks/reviewing/ball-valve-downsize (no GATE_FAST). Expect: 57
lines including ✓ docker-build-environment, ✓ docker-build-tests,
✓ agent-image-isolation, ✓ oracle-runner-reward-aware, ✓ oracle-verifier-roundtrip,
ending GATE-TASK PASS — verified true 2026-08-07.

Run _build/gate-task.sh tasks/reviewing/eir-dc-motor. Expect: same shape, ending
GATE-TASK PASS — verified true 2026-08-07.

Run _build/gate-task.sh tasks/reviewing/pipe-parts-scad. Expect: same shape, ending
GATE-TASK PASS — verified true 2026-08-07.

Final check: all three reviewing-stage units are fully green on the automated gate. Nothing
here explains why they are not yet accepted — per company/NOTES.md the gap is human/SME/Founder
review bandwidth, not a broken build (see F8 for whether that note's own dates are current).

F4

The oracle discriminates a real defect from a cosmetic one

Proves the verifier is not a rubber stamp: it must pass the true solution, fail the unsolved base, fail three deliberate near-misses on exactly the constraint each one violates, fail every measured-property mutation, and still pass a formatting-only change.

Persona / precondition: the pilot unit's cached images present (fbta[openai credential redacted]*).

0/3

Run _build/oracles/ball-valve-downsize/gate-oracle.sh directly (bypassing the gate-task.sh
wrapper). Expect: the run exercises the full battery — primary solution ×3, an alternate
valid profile ×3, a frozen naive probe (fails), the unsolved base state (fails), three sliver
near-miss profiles (each fails exactly one named pytest check — e.g. test_watertightness or
test_exterior_overhang), every measured-property mutation (each fails), and one
formatting-only variation (passes) — closing with the exact line ROUNDTRIP PASS: primary and
alternate solutions passed three times each; the base and frozen naive states failed; all
three sliver near-misses failed exactly one named test; every measured-property mutation
failed; formatting-only variation passed
. Verified true 2026-08-07.

Grep the run's own trace for the reward-file convention rather than trusting exit codes:
Expect lines of the shape mutation-watertightness: container-status=0 reward.txt=0 (a
mutation that fails STILL exits the container cleanly — status 0 — but writes a losing reward)
and format-only: container-status=0 reward.txt=1 for the passing formatting-only run —
proving the Harbor reward contract is judged by the reward FILE's value, never the container's
exit code alone.

Final check: grep -q 'reward\.txt' _build/oracles/ball-valve-downsize/gate-oracle.sh — the
runner itself reads the reward file rather than only checking exit codes (this is exactly what
gate-task.sh's own oracle-runner-reward-aware check enforces).

F5

Verifier isolation: the agent's box cannot see the answer key

Proves the two-image split (environment_mode = "separate") actually holds: the agent's image must never carry solution/ or tests/test*, while the verifier's image legitimately does.

Persona / precondition: fbta[openai credential redacted] and -tests images cached.

0/4

Run docker run --rm --entrypoint sh fbta[openai credential redacted] -c 'find / -maxdepth 4 \(
-name solve.sh -o -name test.sh -o -path "*/solution/*" -o -path "*/tests/test*" \)
2>/dev/null'
. Expect: empty output — nothing found. Verified true 2026-08-07.

Run docker run --rm --entrypoint sh fbta[openai credential redacted] -c 'ls -la /workspace'.
Expect: only the agent's working material (verified contents: data and release
directories) — no solution or tests directory visible.

Run docker run --rm --entrypoint sh fbta[openai credential redacted] -c 'ls /tests' — the
SEPARATE verifier image. Expect: test.sh and test_outputs.py both present — verified
true. This confirms isolation is directional: the verifier legitimately holds the answer key,
the agent provably does not.

Final check: re-read task.toml's [verifier] section — environment_mode = "separate" is
present (this is the contract F5 is proving in practice, not just on paper).

F6

Company ledger drift: NOTES.md's "founding state" is already stale

KNOWN DEFECT found while grounding this plan — write the step that exposes it. company/NOTES.md dated "2026-08-07 — founding state" makes claims that its own sibling files contradict.

Persona / precondition: at ~/Projects/terminus3; read-only.

0/4

Read company/NOTES.md's founding-state entry: "Chief suite DESIGNED-BUT-UNBOOTED. All four
chief seats carry status: UNPROVEN in staffing.yaml... No seat packet in company/seats/ has
been written yet — packets are Step-6 artifacts."
Note the claim before checking it.

Run grep -n "status:" company/staffing.yaml. Expect per the NOTES claim: UNPROVEN on
all five chief roles (ceo + 4 department chiefs). Currently contradicted: every one reads
status: PROVEN-PARKED.

Run find company/seats -type f. Expect per the NOTES claim: no files. Currently
contradicted:
five .packet.md files exist (one per chief token —
terminus3-ceo/.packet.md, terminus3-eng-chief/.packet.md, terminus3-ops-chief/.packet.md,
terminus3-product-chief/.packet.md, terminus3-quality-chief/.packet.md), plus
terminus3-ceo/PARK-terminus3-ceo.md.

Final check: the correct behaviour is that company/NOTES.md is either corrected or dated
per-entry so a reader can tell which sentences are still live — right now, someone trusting the
NOTES snapshot instead of staffing.yaml/company/seats/ directly would believe the company has
not booted its leadership when it demonstrably has (all five chiefs PROVEN-PARKED with packets on
disk).

F7

The kanban board reflects the real fleet of parked chiefs

Proves the board a human actually looks at agrees with the handoff files and staffing roster underneath it.

Persona / precondition: Observify reachable at 127.0.0.1:8787.

0/4

Run curl -s "http://127.0.0.1:8787/api/kanban?project=terminus3". Expect:
"reachable": true and an In Progress column with cards. Verified true 2026-08-07.

Cross-check every chief_token in company/staffing.yaml and every file under _handoff/
against the board's card list. Expect: a /continue <token> card for each of
terminus3-ceo, terminus3-eng-chief, terminus3-ops-chief, terminus3-product-chief,
terminus3-quality-chief, plus the working-session tokens overnight-44,
terminus-batch2, terminus-batch3 — all present, verified true.

Check the other columns. Expect (honest-empty convention): Backlog,
Currently Running, In Development, Testing, Done, and Deployed all report 0 cards
— verified true; nothing has been pushed through those stages yet.

Final check: the board's own "refreshed" timestamp read Jul 28, 2026 while it was still
correctly listing an overnight-44 card whose last activity is Aug 6 — worth confirming with
Observify's own refresh cadence directly (NOT verified here as a defect; flagged as unresolved,
since terminus3-side evidence alone cannot tell staleness from a display label that means
something else).

F8

Revision-idle escalation clock: the claimed dates don't match disk

The Founder-facing escalation trigger in company/CONTRACTS.md §8 depends on an "idle since" date. Confirm it's independently checkable — it currently is not, from files alone.

Persona / precondition: at ~/Projects/terminus3; read-only.

0/4

Read the claim in company/CONTRACTS.md §8 / company/NOTES.md: "downsize idle from 08/04,
eir from 08/05 -> nudge ladder fires ~08/18 (day 14)."

Run find tasks/reviewing/ball-valve-downsize -type f -exec stat -f "%Sm" -t "%Y-%m-%d" {} \;
| sort -r | head -1
. Expect per the claim (last touched 08/04): a date on or before
2026-08-04. Currently reads 2026-08-07 (same day this plan was written).

Repeat for tasks/reviewing/eir-dc-motor. Expect per the claim (last touched 08/05): a
date on or before 2026-08-05. Currently reads 2026-08-07.

Final check: this does not prove the escalation math is wrong — "idle" plausibly means
reviewer/SME silence, not file inactivity, and file writes from a build/audit pass would not
reset that clock. It proves the claim is NOT verifiable from file mtimes alone, so the ~08/18
nudge date should be re-derived from whatever signal "idle" actually means (last reviewer comment,
not last file write) before anyone treats day 14 as a hard trigger.

F9

Payout-taxonomy correctness across the corpus

Proves every delivered unit is tagged the way the mission and the platform's payout math both assume, and derives (without asserting as platform-confirmed) what full acceptance would be worth.

Persona / precondition: at ~/Projects/terminus3; read-only.

0/3

[sensitive content omitted]
[sensitive content omitted]
one of the 71 units correctly tagged. Verified true 2026-08-07.

Count OpenSCAD-bonus-eligible units — metadata.languages containing openscad — across all
71 task.toml files. Expect: 60 of 71. Verified true 2026-08-07.

Final check: per company/CHARTER.md's stated payout ($300 flat + $50 OpenSCAD bonus per
accepted task), if all 71 units were accepted as-is the derived gross would be
71 × $300 + 60 × $50 = $21,300 + $3,000 = $24,300 — stated here as an arithmetic derivation from
the corpus + the charter's own numbers, explicitly NOT a platform-confirmed acceptance figure
(only 1 of 71 has actually been accepted to date).

F10

No-touch surfaces are present and untouched

Proves the hard out-of-scope list in company/CONTRACTS.md §2 / the project constitution's P-IV is still intact — this flow reads only; it must never be the flow that touches them.

Persona / precondition: at ~/Projects/terminus3; read-only, no edits.

0/6

Run ls -la stlFiles.zip. Expect: present, ~2.7 GB, Jul 31 timestamp.

Run ls -ld extracted. Expect: present.

Run ls -ld think-like-fable. Expect: present.

Run ls -la "Peer Coaches Frontier Bench Spec Doc .md" (note the double space before .md
in the real filename). Expect: present, 17318 bytes, Jul 29 timestamp.

Run ls -la _build/inventory/manifest.yaml. Expect: present.

Final check: none of the five were modified running this test pass — re-run step 1's ls -la
and confirm the size/timestamp is unchanged from what was recorded when this plan was authored
(do not touch, edit, or delete any of them — CONTRACTS §2 hard no-touch, project constitution
P-IV).

Not covered and why

  • The other 60 "working"-stage units' full landing gates. F1 covers Phase A (structural) for

all 71 units; Phase B (docker + oracle) was spot-checked only on the 3 reviewing/ units plus
the 1 accepted/ unit — running docker+oracle on all 63 working/ units is a real verification
that should happen before the next promotion wave, just not inside one authoring pass here.

  • Platform submission / upload flow. Founder-only, irreversible (CONTRACTS §5) — not

something this plan should exercise even read-only-adjacent.

  • Slack/comms to Justin or Snorkel. Founder-only per CONTRACTS §5; no test surface.
  • agy / grok / ollama seat live probes. staffing.yaml marks these probe_verified: false

with a dated warning — a real probe run is needed before they're staffed, but that's a live
dispatch action, not a read-only verification.

  • stb harbor invocation and the digit-agreement scan. company/NOTES.md itself calls these

"practice-not-script gates" — conductor practices, not committed scripts. Nothing exists yet to
run; verify the actual stb command against the installed tool before the next audit cycle.

  • Docker VM memory-ceiling flips / any host mutation. CONTRACTS §5 marks host/Docker settings

changes Founder-executed only.

  • Multi-machine dispatch. CONTRACTS §2: "this Mac only — amd64 digest pins forbid the Pi." No

second machine to test against.

  • The from-scratch CPO sourcing/license-gate pipeline. company/NOTES.md: "UNPROVEN as a

platform-acceptance path — no from-scratch task has ever been submitted." Nothing shipped to
test yet.

Results

FlowVerdictFailures → tickets
F1 structural gate corpus-wide
F2 fleet ground-truth counter
F3 full landing gate (3 reviewing units)
F4 oracle discrimination
F5 verifier isolation
F6 company ledger drift
F7 kanban board vs fleet
F8 revision-idle clock
F9 payout taxonomy
F10 no-touch surfaces
updated just nownext 3m 00s