Load the site. Expect: brief Loading…, then the sign-in card: heading ▸ Sign in to, fields
AgentifyEmail, Passphrase, Invite code (optional).
`
agentify
Derived from: commit b5fe60e (dev), 2026-07-25, against the production deployment at
Flows
check each step and final checkSign in, wrong passphrase first
Proves the identity boundary: deny-by-default, no account enumeration, then a clean entry.
Persona / precondition: allowlisted user with a set passphrase, signed out, fresh tab on https://agentify.thewolf.tech.
Enter your email, a wrong passphrase, click Sign in. Expect: error text exactlyinvalid credentials — no hint whether the email exists. `
Enter a non-existent email and any passphrase, click Sign in. Expect: the sameinvalid credentials — byte-identical to step 2. `
Enter your email with the correct passphrase, click Sign in. Expect: the session
console at #/: header shows the Agentify wordmark, your email, and your role pill
(god/orchestration/trial), tabs SESSION / MARKETPLACE, buttons Tour and Sign out.
`
Final check: reloading the page keeps you signed in (session cookie survives a refresh).
`
---
First-visit product tour
Proves a stranger's first two minutes: the tour auto-opens, demos safely, and exits cleanly.
Persona / precondition: a browser profile that has never visited the site (or clear the localStorage key agentify.tour.seen.v1), signed in.
Arrive at #/ for the first time. Expect: the tour opens by itself: panel titledProduct tour, step counter 1 / 17, a Demo mode badge in the header, and a bannerScripted product walkthrough with a Demo data badge. `
Click Next through at least 5 steps. Expect: the step counter advances, each step
highlights a different area (Welcome, Access, Billing, Credentials, Kit Atlas, …)
and the demo pane changes with it. Nothing you click in the demo spends money or touches a
real box (the demo console says so explicitly). `
Press Esc. Expect: the tour closes; the Demo mode badge disappears; the real
console is back. `
Click Tour in the header, then click Skip tour. Expect: it reopens on demand and
dismisses again. `
Final check: reload — the tour does not auto-open a second time.
`
---
Start a free-trial session (blank)
Proves the core promise: a real cloud session with no key, no account, no spend.
Persona / precondition: signed in; no session currently active.
On #/, find Start a session. In the kit picker, leave Blank session (no kit) selected
(it shows selected). Expect: Model reads @cf/openai/gpt-oss-20b; Billing readsFree trial — no key needed (runs on agentify's Cloudflare allowance); Storage readsEphemeral — fresh box, zip in/out. `
Check the trial readout under Billing. Expect: Shared demo pool: N / 10,000 Neurons plus your own allowance — real numbers, not a loading spinner stuck forever.
remaining
`
Click Start session. Expect: the confirm bar: This provisions a cloud box and with Confirm & start / Cancel.
reserves shared demo allowance. Continue?
`
Click Confirm & start. Expect: within ~15s a session card appears: 8-char id, model,ephemeral, status pill turning ready (green), ticker ending terminal live, telemetryttyd attached, and the Codex boot box in the terminal (OpenAI Codex, the model, a/data/workspaces/sess-… directory). No red error banner. `
Final check: the panes below the terminal render their honest empty states: Fleet showsNo fleet reported yet…, Live agents shows one of its three empty-state messages, Workspace
shows workspace empty — import files to get started. `
---
Real work in the session (base kit)
Proves the terminal is a working agent, the base kit is present, and no internal tooling vocabulary leaks.
Persona / precondition: F3's session live.
Click into the terminal and type ls ~/skills ~/.agents/skills, press Enter. Expect: both
directories list exactly: begin continue dogfood ground handoff test-it.
`
Type $begin and press Enter. Expect: within ~90s (whole turns render at once on the
trial model) the agent responds with a short interview — 3-5 questions about your goal — in
plain product language. It must NOT mention any internal kit name, ask where a kit root is,
or reference directories like kernel/. `
Answer the questions with a small project (e.g. "a CLI that counts words in a file; done =
it runs on a sample"). Expect: the agent creates files (a PLAN.md, source files) and
RUNS shell commands visibly — no sandbox errors (bwrap…), no approval prompts, noinvalid_prompt / 429 red JSON. Slow is fine; silent death is a fail.
`
When it reports the slice built, check the Workspace pane. Expect: the file count is
> 0 and PLAN.md appears in the listing. `
Final check: ask it run the project and show me the output — it executes and shows real
program output that matches what was built. `
---
/test-it inside the session
Proves the newest base skill: the agent writes a user-testing plan for what it just built.
Persona / precondition: F4 completed in the same session.
Type $test-it and press Enter. Expect: the agent produces USER_TESTING.md in the
workspace: flows derived from the actual project, numbered steps, an Expect: per step,
pass/fail record lines, and a "not covered" section. `
Check the Workspace pane. Expect: USER_TESTING.md is listed with a non-zero size.
`
Final check: read one flow — you (a human) could execute it cold without asking the agent
anything. `
---
Marketplace kit → seeded session
Proves the marketplace-to-session bridge: browse, pick, and the kit's intake seeds the box.
Persona / precondition: signed in; end any active session first (F7 step 4 shows how).
Click MARKETPLACE. Expect: title Marketplace, stat tiles Kits / Verticals /Tags with non-zero numbers, vertical filter chips starting with all, and kit cards.
`
Type a term in Search kits by title, vertical, tags… that matches a kit you can see.
Expect: the result count line (N of M kits) drops and only matching cards remain.
`
Click a kit card. Expect: the kit page: ← Marketplace back link, How it runs diagram,Skills list, Intake cards, a /begin prompt block with a copy button, and the button
Use this kit →. `
Click Use this kit →. Expect: back on #/, the wizard's kit picker has this kit
pre-selected (accent border, selected), and the detail box says This kit's /begin intake `
will be seeded into your workspace as START-HERE.md.
Click Start session, then Confirm & start. Expect: the session boots (as F3.4)
and START-HERE.md appears in the Workspace pane listing. `
Final check: in the terminal, cat START-HERE.md shows the kit's intake text matching the
kit page's /begin prompt. `
---
Session lifecycle: reset, wipe, end, single-active
Proves the lifecycle controls and the one-session-at-a-time rule.
Persona / precondition: a live session with at least one file in the workspace (F4 or F6).
Click Reset. Expect: a browser confirm: Recreate the box and keep your workspace? Accept it. The card shows
This spends against your cap.Resetting…, then returns toready — and the Workspace pane still lists your files. `
Click Wipe. Expect: confirm Erase the workspace and start clean? This spends against Accept. After
your cap.Wiping… → ready, the workspace is workspace empty — import (files gone). `
files to get started
Open the site in a second tab and try to start another session (Start session → Confirm &
start). Expect: A session is already active. Refreshing… and the tab shows the existing
session instead of a duplicate. `
Click End session. Expect: after Ending…, back on the wizard with the noteSession ended. Accrued $X.XX against your cap. (a number, even $0.00).
`
Final check: the idle console aside shows sessions none active and Awaiting commission. `
Start a session to bring a cloud box online.
---
Workspace export and import
Proves work leaves and enters the ephemeral box intact.
Persona / precondition: a live session with 2+ files (re-run F4 briefly, or import in step 3).
In Workspace, click Export .zip. Expect: the browser downloadsagentify-<sessionId>.zip; opening it shows the same files the pane lists.
`
Click Wipe (accept the confirm). Expect: workspace empty — import files to get. `
started
Click Import files and choose the zip from step 1 (or drag it onto the listing box —
it should light up on dragover). Expect: Importing…, then a green imported N files
and the file rows return with sizes. `
Final check: in the terminal, ls shows the restored files and one of them opens withcat showing its original content. `
---
Vault a credential, run vaulted billing
Proves the encrypted vault and the bring-your-own-key path.
Persona / precondition: signed in, no active session; a real API key you're willing to store (e.g. Anthropic or OpenAI).
Scroll to Connect model account at the bottom of #/. Expect: status Not connected,
a Credential type select listing providers (OpenAI API, Anthropic Claude, xAI Grok,Google Gemini, Cloudflare Workers AI, Local (Ollama), …).
`
Pick a provider, paste the key into Secret, click Connect. Expect: the status turns
green: Connected — <provider label>, GitHub no, updated just now. The key itself is never
redisplayed anywhere. `
In the wizard set Billing to Vaulted key — bill my connected credential. Expect: the
Model dropdown now shows that provider's model family, with the helper Using the connected. `
<kind> provider…
Start + confirm a session, and in the terminal ask the model something trivial. Expect:
a normal boot and a real answer (billed to your key, not the trial pool).
`
End the session; click Disconnect in the credentials panel. Expect: status returns toNot connected. `
Final check: with nothing connected, choosing Vaulted key billing and starting a session
is refused with Connect the required credential, then start a session.
`
⚠ Known gotchas for this flow (found 2026-07-25, real):
- Use an OpenAI key with full permissions. A restricted/read-only key reaches OpenAI fine but
the session dies with 401 … Missing scopes: api.responses.write. Create the key with All
permissions (or Restricted + Responses→Write). The same error also appears if your org/project
role is Reader rather than Writer/Owner.
- Only OpenAI and Azure are realistically usable today. Every vaulted provider is configured
with wire_api="responses", but Anthropic, Gemini, OpenRouter, and Ollama serve
chat-completions, not /responses — Codex removed wire_api="chat", so those families cannot
work without a translating bridge (Cloudflare has one; the others do not). ⚠ Reasoned from the
providers' APIs, not yet tested per-provider — worth a dedicated pass.
---
Native Codex login
Proves the primary power-user mode: your own subscription inside the terminal.
Persona / precondition: signed in, no active session; an OpenAI/ChatGPT account.
Set Billing to Native login — sign in inside the terminal, bills my subscription.
Expect: the How native Codex login works guide expands (steps 01/02/03, a codex login
snippet with a Copy button) and the Model list changes to the OpenAI family
(gpt-5-codex, gpt-5, o4-mini). `
Start + confirm the session, choose 1. Sign in with ChatGPT, press Enter, and open the
URL Codex prints. Approve it. Expect: your browser lands on a localhost:1455 page saying
it can't be reached — this is expected, and the address bar holds your sign-in code.
`
Copy that whole address. Above the terminal, paste it into Signing in? Finish it here
and click Finish sign-in. Expect: Signed in — restarting your terminal… followed bySigned in. Your terminal is ready. `
Look at the terminal — it should have relaunched by itself. Expect: a working Codex prompt,
signed in, with no sign-in menu and no manual Reset needed.That link did not complete the sign-in — it is probably from an earlier attempt…
⚠ If you paste a URL from an EARLIER sign-in attempt, the expected result is the refusal, NOT aesc
success message. Worth testing deliberately: start a sign-in, cancel it with , start a
second, and paste the FIRST one's URL.
Type a prompt (e.g. say hi). Expect: a real answer on gpt-5-codex, billed to your
subscription — no 429, no invalid_prompt. `
[sensitive content omitted]
Expect: a refusal, not a fetch — the relay accepts only loopback callback URLs.
`
Two-user check (needs Jan). With your native session live and signed in, have Jan start a
native session, begin a sign-in there, and paste HIS callback URL into HIS session.
Expect: Jan's sign-in either completes in his own session or is refused — it must never
land in yours. Then check your terminal: it must still be signed in as you.
`
Wrong-session paste. Paste a callback URL for a port your session does not own (e.g. any
unrelated local port). Expect: no sign-in from this session is listening on that port —
the relay proves ownership rather than trusting the session id you named.
`
Final check: end the session, start a fresh trial session: it must NOT inherit your
native login (each session's auth is isolated — a fresh box knows nothing of your account).
`
---
Trial guardrails (conditional)
Proves the money guard refuses instead of overspending. Only executable when the shared pool is actually low/exhausted — do not force it; check it on a busy day.
With the daily pool exhausted, click Confirm & start on a trial session. Expect: the
refusal The free daily pool is used up. It resets at 00:00 UTC. — and no session appears.
`
⚠ Known gotcha (observed 2026-07-25): the "resets at 00:00 UTC" copy is optimistic —
Cloudflare's allocation was still exhausted at 00:46 UTC. The refusal is honest; the reset time
in the message is not something agentify can guarantee. Consider softening the copy or deriving
the reset from upstream.
---
God-kit overlay (god account only)
Proves the role gate and the overlay superset.
Persona / precondition: signed in as the god account; no active session.
Look at the wizard as the member account first (sign out/in as needed). Expect: noAdd the god kit checkbox exists for a non-god role. `
As god: check Add the god kit, start + confirm a trial session. Expect: session boots;ls ~/skills | wc -l in the terminal prints 40 (the full set), and the six base skills are
among them. `
Final check: $backlog (a god-overlay skill) is recognized by codex as a command.
`
---
Phone + notifications (real handset)
Proves the mobile surface — the known gap: it has only ever been tested in a synthetic viewport, never on the owner's device.
Persona / precondition: your actual phone, on the live site, signed in.
Load the site and sign in on the phone. Expect: no horizontal scroll anywhere; the tour
(if it opens) is a full-screen single-column dialog labeled Demo first · step details below.
`
Start a trial session and type into the terminal. Expect: the keyboard doesn't cover the
input; the session card and panes stack single-column and stay usable.
`
In Mobile notifications, click Enable notifications and accept the permission prompt.
Expect: chip turns enabled, message Notifications enabled on this device.
`
Click Send test. Expect: Test sent to 1 device(s). and a real push arrives on the
phone. `
Final check: end the session from the phone (End session reachable and working).
`
---
Error paths sweep
Proves the product fails loudly and honestly at its edges.
GitHub storage without a token:[redacted] set Storage to GitHub — persistent workspace via, enter any repo URL, start + confirm. Expect:** refusal
GitHubConnect the required with the hint about the
credential, then start a session.Connect model account panel.
`
Trial signup while closed: signed out, enter an email and click Start a free trial.
Expect: self-serve signup is not open yet. Enter an invite code above.
`
Marketplace search with gibberish: search zzzzzz. Expect: No kits match your — not an empty white area. `
search.
Diff pane without git: in a session whose workspace has no repo, expand Code review.
Expect: workspace is not a git repository — run git init or clone a repo in the. `
terminal
---
Not covered and why
- GitHub persistent storage end-to-end (clone→work→push): needs a vaulted GitHub token and a
sacrificial repo. Planned, not scripted here.
Keepstorage across teardown: needs two sessions separated by a deliberate teardown;
semantics should be scripted once the first pass of this document is green.
- Multi-user concurrency (two people, two sessions, isolation between them): needs a second
tester (Jan) — sessions must never see each other's workspaces or auth.
- Trial overage / midnight reset: crossing 00:00 UTC and CF billing edges — observed
passively, not forced.
- Observify pane live mode: server-side URL is deliberately unconfigured; only the disabled
empty-state is testable (F3 final check).
- Slack / Telegram notifications: operator tokens unset in prod.
- Operator CLI (
agentify setpass, deploy lanes): server-side operations, not user flows. - iOS push specifics: F13 covers one handset; iOS PWA push behaves differently and needs its
own pass if an iPhone is the daily driver.
Results
| Flow | Verdict | Failures → tickets |
|---|---|---|
| F1 sign in | ||
| F2 tour | ||
| F3 trial session | ||
| F4 real work | ||
| F5 /test-it | ||
| F6 marketplace seed | ||
| F7 lifecycle | ||
| F8 export/import | ||
| F9 vault | ||
| F10 native login | ||
| F11 trial guard | ||
| F12 god overlay | ||
| F13 phone | ||
| F14 error paths |