Observify

User Tests · wolfshelf-calibre
← all user tests
USER TEST PLAN

wolfshelf-calibre

Derived from: no VCS — this project is confirmed NOT a git repository (see F1.1), so there is

0/38 checked · 0 pass · 0 fail

Flows

check each step and final check
F1

Project identity and board hygiene

Proves the project's own kanban and its directory agree with reality: no VCS, no README, and the board's backlog matches what's actually stalled here.

Persona / precondition: an operator with a terminal at ~/Projects/wolfshelf-calibre and network access to 127.0.0.1:8787.

0/5

Run git -C ~/Projects/wolfshelf-calibre status. Expect: the exact line fatal: not a git
repository (or any of the parent directories): .git
— confirmed this session; this project has
no VCS by design (per _handoff/wolfshelf.md: "not a git repo, no VCS").

Run find ~/Projects/wolfshelf-calibre -maxdepth 1 -iname "README*". Expect: no output —
there is no root README, matching the kanban's own baseline card text (see step 4).

Run curl -s "http://127.0.0.1:8787/api/kanban?project=wolfshelf-calibre". Expect: the JSON
contains "reachable": true, and among its columns the ids in_progress, running,
in_development, testing, done, deployed each report zero cards; only backlog is
non-empty.

Inspect the backlog column's two cards. Expect: one card named /continue wolfshelf
sourced from _handoff/wolfshelf.md, and one card named Repo baseline whose raw text is
Repo baseline — wolfshelf-calibre project directory (no root README found).

Final check: run find ~/Projects/wolfshelf-calibre -not -path "*/think-like-fable/*" -not
-path "*/.claude/*" -type f -newer ~/Projects/wolfshelf-calibre/_handoff/wolfshelf.md
. Expect: no
output — outside the shared kit mirror and Claude settings, nothing in this project has changed
since the 2026-07-19 handoff was written. The project is exactly as stalled as its own board says.

F2

Wishlist arithmetic checks out against its own source JSON

Proves the headline wishlist numbers in wishlist.md are computed from series_gaps.json, not asserted by hand.

Persona / precondition: an operator with python3 and read access to wishlist.md and reports/series_gaps.json.

0/5

Run grep -n "series gaps" -i wishlist.md or open the file and read the "Full machine-readable
list" line. Expect: the exact substring `reports/series_gaps.json` (148 series · 449
missing volumes)
.

Run python3 -c "import json; print(len(json.load(open('reports/series_gaps.json'))))" from
~/Projects/wolfshelf-calibre. Expect: 148 — matches step 1's series count exactly.

Run python3 -c "import json; d=json.load(open('reports/series_gaps.json')); print(sum(len(s['missing']) for s in d))".
Expect: 449 — matches step 1's missing-volume count exactly.

Run python3 -c "import json; d=json.load(open('reports/series_gaps.json')); s=[x for x in d if len(x['missing'])==1]; m=[x for x in d if len(x['missing'])>1]; print(len(s), len(m))".
Expect: 66 82 — matching wishlist.md's claims of "66 series need exactly ONE volume" and
the "82 of them" multi-gap series named under "Larger gaps worth a look".

Final check: 66 + 82 == 148 — the single-gap and multi-gap counts from step 4 partition the
full series list from step 2 with nothing left uncategorized.

F3

U13 bulk-enrichment output integrity (Mac-side artifacts)

Proves the enrichment pass that ran against the 2,938 bare-with-text books actually completed 1:1 with no dropped books and no corrupt cover files — the part of U13 that IS checkable from the Mac.

Persona / precondition: an operator with python3 and read access to enrich/enrich_bundle.jsonl, enrich/enrich_results.jsonl, enrich/covers/.

0/6

Run wc -l enrich/enrich_bundle.jsonl enrich/enrich_results.jsonl from the project root.
Expect: both files report exactly 2938 lines — every book that went in produced a result.

Run a python3 pass counting lines with a truthy isbn key in enrich_results.jsonl.
Expect: 2001 / 2938 (68%) — matching _handoff/wolfshelf.md's claim.

Run the same pass counting lines with cover: true. Expect: 1281 / 2938 (44%) — matching
the handoff's claim.

Run ls enrich/covers | wc -l, then verify programmatically that the set of ids with
cover: true in the JSONL equals exactly the set of <id>.jpg filenames in enrich/covers/
(no id missing a file, no file without a matching flag). Expect: 1281 files, zero mismatches
either direction.

Run find enrich/covers -size 0 | wc -l. Expect: 0 — no zero-byte/corrupt cover files.

Final check: run file enrich/covers/10442.jpg (or any id from the listing). Expect: output
containing JPEG image data — the file is a real image, not a stub.

F4

KNOWN DEFECT: U13 output schema silently dropped four planned fields

Exposes a real, currently-unresolved bug named in the project's own handoff: the plan promised seven derived fields per book; the shipped output only ever wrote five.

Persona / precondition: read access to PROJECT.md and enrich/enrich_results.jsonl.

0/4

Read the U13 row in PROJECT.md. Expect (the plan, as written): the pipeline derives
title/author/description/subjects/category/isbn(if in text)/is_junk per book, with
is_junk → quarantine.

Run python3 -c "import json; keys=[redacted]; [keys.update(json.loads(l).keys()) for l in open('enrich/enrich_results.jsonl')]; print(sorted(keys))".
CORRECT/expected behavior per the plan in step 1: the key set should include title,
author, category, and is_junk. Actual, verified this session: the key set is exactly
['cover', 'description', 'id', 'isbn', 'subjects']title, author, category, and
is_junk are absent from every one of the 2,938 results. This currently FAILS — mark it
fail, this is not a step to pass.

Run grep -c "is_junk" enrich/enrich_results.jsonl. Expect if the field had ever been
emitted (even as always-false):
at least one occurrence. Actual: 0 — the string
is_junk does not appear anywhere in the file. The junk-detection half of the pipeline was
never emitted, not "run and found nothing." Currently FAILS.

Final check: because of this defect, U12 (truly-bare quarantine) cannot safely treat "no
is_junk: true found" as "no junk exists."
Record that dependency here rather than letting U12
proceed on a false all-clear — this is the exact risk _handoff/wolfshelf.md flags under "Schema
drift, unexplained."

F5

KNOWN DEFECT: PROJECT.md's U13 row is stale against its own data

Exposes a live documentation-drift bug: the ledger still describes a 4-book pilot although the 2,938-book bulk run finished and has been sitting on disk since 2026-07-18.

Persona / precondition: read access to PROJECT.md and enrich/enrich_results.jsonl.

0/3

Read the tail of the U13 row in PROJECT.md. Expect if current: a description of the full
2,938-book bulk run (counts, hit rate, cover yield). Actual, verified this session: the row
ends with the literal text Pilot: 4 books enriched, real covers found for 3/4 (Inge,
Énard/Goncourt, Yablo/Princeton — all correct), Ahriman=coverless tail.
— no mention of the
completed bulk pass anywhere in the row. Currently FAILS, and is still true as of this run
(not something that was already fixed since the handoff was written).

Run stat -f "%Sm %N" PROJECT.md enrich/enrich_results.jsonl. Expect if PROJECT.md were
current:
PROJECT.md's mtime at or after the results file's mtime. Actual: PROJECT.md is
Jul 17 18:17:07 2026; the results file is Jul 18 01:20:01 2026 — PROJECT.md predates the
very data its U13 row is supposed to summarize by about 7 hours. Currently FAILS.

Final check: _handoff/wolfshelf.md already names this exact gap as its own NEXT STEP #3
("Update PROJECT.md's U13 row to reflect the completed 2,938-book bulk run") — the plan and the
ledger agree on what's broken, it just hasn't been fixed yet. Confirm that NEXT STEP #3 is still
present and still unaddressed.

F6

The Mac/Pi machine boundary is real, not a convenient excuse

Proves what this session can and cannot verify is an honest limit, not a shortcut — and that hitting the limit doesn't tempt a shortcut like silently trusting an unverified host key.

[sensitive content omitted]

0/4

Read think-like-fable/kernel/machines.yaml. Expect: it is the shipped TEMPLATE only — no
real Pi hostname, IP, or user; its own resolution ladder ends in "ask the human — never guess
hostname variants"
.

Run ssh -o BatchMode=yes -o ConnectTimeout=3 wolfplex "echo ok". Expect: report whatever
actually happens rather than assuming success or failure. This session got Host key
verification failed
— a host answers at that alias, but its key is not trusted from this
session, which is neither a clean reach nor a clean absence. Do not pass `-o
StrictHostKeyChecking=[redacted] or otherwise accept the new key to force a result — that decision
belongs to a human, not this test.

Because of step 2, treat every Pi-side claim in PROJECT.md (U1–U3, U5–U6, U9–U10, and the
metadata.db writeback question raised in F4's final check) as not independently re-verified
this session
— the same boundary _handoff/wolfshelf.md already declares for itself.

Final check: confirm no command in this flow used a trust-deciding flag
(StrictHostKeyChecking=[redacted], key acceptance, sudo`, or any write) — the boundary held even under a
failed connection.

F7

Generated-surface parity between CLAUDE.md and AGENTS.md

Proves the project's two agent-instruction files are one generated artifact under two names, not two hand-edited copies quietly drifting apart (Article X).

Persona / precondition: read access to CLAUDE.md and AGENTS.md in the project root.

0/3

Run diff ~/Projects/wolfshelf-calibre/CLAUDE.md ~/Projects/wolfshelf-calibre/AGENTS.md.
Expect: the only reported difference is the first line's generator-source comment
(CLAUDE-block.md vs AGENTS-block.md) — zero differences anywhere else in the file.

Read the first two lines of either file. Expect: both contain the literal text DO NOT EDIT
(Art. X)
.

Final check: read the span between <!-- BEGIN PROJECT-ARTICLES --> and <!-- END
PROJECT-ARTICLES -->
in CLAUDE.md. Expect: it contains only the comment filled by /begin —
hand-authored; the generator never enters this span
and no actual per-project articles — this
project's constitution has never been instantiated, matching "unfilled templates govern nothing."

F8

Credential hygiene in the project's own Claude settings

Proves the workspace that drove this whole migration didn't turn itself into a place secrets leak from.

Persona / precondition: read access to .claude/settings.local.json and the project tree.

0/4

[sensitive content omitted]
Read(secrets/**), and Bash(sudo:*).

[sensitive content omitted]
[sensitive content omitted]
in the Mac-side project tree.

Run grep -rlE "AIza|sk-[A-Za-z0-9]{10}|ya29\.|Bearer [redacted]*.md" --include="*.jsonl" --include="*.json" -not -path "*/think-like-fable/*" (adjust flags for
your grep). Expect: no matches — the rclone Google OAuth token this project depends on lives
in the Pi's own rclone config, never copied into this Mac-side directory.

Final check: note that "defaultMode": "bypassPermissions" is present in the same settings
file. This is a deliberate, already-made project choice, not something this plan is adjudicating —
record it in the notes column but do not fail the flow over it.

F9

reMarkable access: documentation cross-check (liveness unverifiable from Mac)

The project's headline promise — "open the library to the reMarkable" — checked for internal consistency across its own docs; whether the sync is still actually firing today is out of reach per F6.

Persona / precondition: read access to reports/remarkable-access-memo.md and PROJECT.md.

0/4

Read reports/remarkable-access-memo.md. Expect: it recommends Option A — Google Drive
integration (native, zero risk)
as the starting point, naming Option B (rmapi cloud push)
as a later "Holt tie-in," not the current path.

Read PROJECT.md's U10 row. Expect: it claims Option A is LIVErclone configured on the
Pi against a gdrive: remote, shelf-sync.sh + shelf-add.sh scripts, and a systemd
shelf-sync.timer firing every 15 minutes — i.e. the same option the memo recommended, not a
different one quietly substituted in.

Run find ~/Projects/wolfshelf-calibre -iname "shelf-*". Expect: no output — shelf-sync.sh
and shelf-add.sh are Pi-side artifacts; there is nothing on the Mac to independently inspect.

Final check: because of F6's boundary, this flow cannot confirm the shelf-sync.timer is still
firing today. Record that explicitly — do not let PROJECT.md's LIVE verdict stand in as current
verification.

Not covered and why

  • Pi-side ground truth for U1–U3, U5–U6, U9–U10 (NVMe library integrity, quarantine state,

WolfTools filesystem, rclone/systemd liveness): needs a trusted SSH path to the Pi that this
session does not have (F6). Re-run this section once a human resolves the host-key mismatch.

  • metadata.db writeback for the 2,938 U13 results: _handoff/wolfshelf.md already flags this

as unconfirmed — no script, log, or DB diff exists on the Mac showing the enrichment results
were ever written into the live Calibre database on the Pi. Needs Pi-side inspection.

  • U12 (truly-bare quarantine): not started per PROJECT.md; blocked in practice by F4's schema

gap until is_junk detection is either confirmed intentionally absent or re-run.

  • U14 (wishlist regeneration): needs analyze_library.py, which is Pi-side and not present in

this Mac directory; F2 verifies the current wishlist's internal math, not whether it's the
latest possible wishlist.

  • Textbook edition-upgrade teacher pass: explicitly flagged PROPOSED, awaiting Wolf's go, not

started — nothing to test yet.

  • The 3,155-vs-2,938 bare-with-text discrepancy: PROJECT.md's U13 plan cites 3,155 candidate

books; the bulk run processed 2,938. This may be explained by Tier B resolving ~217 of them
first (removing them from the "bare" pool) — plausible but NOT verified this session; worth a
dedicated check before treating either number as authoritative.

Results

FlowVerdictFailures → tickets
F1 board hygiene
F2 wishlist math
F3 enrichment integrity
F4 schema-drift defect
F5 stale ledger defect
F6 Mac/Pi boundary
F7 generated-surface parity
F8 credential hygiene
F9 reMarkable doc cross-check
updated just nownext 3m 00s