{
 "model": "claude-haiku-5-5",
 "category": "context",
 "scenarios": [
  {
   "id": "c1-handoff",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "ARCHITECTURE.md was written in session 2 (142 lines, 11 modules, data flow, three risks; all 121 backticked names exist in engine/). The driver's 3-follow-up cap ended the run while session 4 still had corrections open, so no final reply went to the user.",
   "notes": "Four sessions and four handoffs, exit codes all 0. Read-before-handoff ran in all four. Sessions 1-3 replied with text only, ran no tools and were not cut off; the text starts after a blank line in all three saved interrupted messages, and those results count as read. In session 4 the cap was about 3,593 tokens and the reply had no text, so a system frame said so, the conversation was restored without the note, saved contextStatus stayed at the measured 56,151, and the handoff marks those results as received after the last reply. Stalls were 1, 0, 1, 2, correct because only session 2 changed a file. Progress check appears only in sessions 2 and 4 with the right counts, and no sub-agent request appears because the count never reached 3. Binding rule leads every continuation and <request> keeps the original text. Old sessions are saved as interrupted and linked by path and id. Pairs match, files are mode 600, and cost recomputes exactly ($0.074266535, 21 requests). Model behaviour: after the document existed, sessions 3 and 4 kept re-verifying it and re-read ARCHITECTURE.md and source ranges despite the seed's instruction not to re-verify. Session 2's handoff did list verification as open and the seeds were accurate, so this is not a defect. Session 4 found real errors in the draft that remain uncorrected in the file (model accuracy, cut short by the driver cap). Session 1 has no session-memory file; this looks content-driven (it ran only Glob and FileRead), though I did not read the session-memory code."
  },
  {
   "id": "c2-compact",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "ARCHITECTURE.md was completed (16,783 bytes, all 11 modules, all 129 backticked names exist in engine/). The turn completed with one context-full question, three compactions, three old-result clearings and no API errors.",
   "notes": "Context-full was asked once and later compactions ran without asking. The request stays pinned word for word beside the summary. Saved contextMessages pair correctly. The one tool failure was the model's own FileEdit (edit 20 of 21 not found, all or nothing, correctly reported), and the retry succeeded. Model behaviour: it re-read files several times after clearing and compaction, and its claim that reads were cut at about 10k characters is not a tool limit; frames show results up to about 31k. I think this came from clipped kept results, inferred from compact.ts and not confirmed in a saved request. Cost recomputes exactly ($0.047807175, 16 requests)."
  },
  {
   "id": "c3-stop-then-continue",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Stop ended turn 1 with a clear error and the turn was saved as interrupted. /compact succeeded (46,898 to 16,631 tokens, archive kept) and the continuation finished ARCHITECTURE.md with no API 400.",
   "notes": "Saved summary has parts 1-3 of 3, each carrying the user's real goal rather than the engine's stop note. The hooks.ts read split across parts keeps the file name and exact ranges (1-186, then 187-274), which fits the labels and continuation note in session/compact.ts. Limit of this check: the raw part inputs are not persisted, so label and '[this block continues in the next part]' text were confirmed from code plus the summary, not read from a saved request. Long-path labels were not exercised because paths are about 120 characters, under the 160 limit. Model wording: parts 1 and 2 still say things like 'no FileRead results appear in this part' and list next steps for other parts, despite the preface; the joined summary stays accurate and the continuation read only queryEngine.ts 515-967 plus a check. The final reply states that lines 1-514 relied on notes. The '93% full' wording counts the 9,000 reserve (46,819 + 9,000 of 60,000). Port 8990 had no listener and leftovers are empty. Cost recomputes exactly ($0.018850325, 11 requests)."
  },
  {
   "id": "c4-handoff-chain",
   "status": "fail",
   "product_defects": [
    {
     "severity": "low",
     "title": "Skill trigger fires on previous session's handoff notes inside a continuation, and the engine tells the model the user's message matched",
     "evidence": "runs-haiku-r17e/c4-handoff-chain/home/.gcode/projects/*/c05cc295-85a8-4313-8720-6496feca148e.json: contextMessages[0].wire.content starts with a <context> block reading 'The user's message matches the trigger of the skill \"browser-verify\"' (4,343 characters before the message). The only match in messages[0].content is at offset 4351, inside <handoff>, where the previous session's notes describe systemPrompt.ts as having 'BrowserVerify guidance'. The <request> does not contain it. The next handoff, .gcode/handoffs/2026-11-05T05-04-55-c05cc295.md, records 'The browser-verify skill does not apply.'",
     "location": "snapshot10z/src/cli/slashCommands.ts:114 (called on the whole continuation from snapshot10z/src/cli/repl.ts:433)",
     "repro": "Have a session's handoff notes mention BrowserVerify or BrowserPlan, for example by reading a source file that names them, then send handoffContinuation(...) to the new session. turnInvocations tests the whole message, so the skill block loads although the user's request never matched. Testing only the quoted request (continuationRequest) would match the rule already used for stalls, where the notes cannot set the value."
    }
   ],
   "task_outcome": "Chain ran as designed: 4 sessions, no lost work, SUMMARIES.md grew from 6 to 7 unique blocks with no duplicates. The task was unfinished at the driver's cap (7 of 22 files), which G-Code did not cause.",
   "notes": "Confidence in the defect: high on the mechanism, moderate on classification. The brief calls a matching message showing the skill once the feature, but here the words came from engine-carried reference notes and the block's claim about the user's message is false. Impact was small: about 1,100 extra tokens in a 40K window and one line in the next handoff; the model ignored the skill and no wrong action followed. All other handoff checks passed. Session 1's read-before request ran with a 1,676-token cap (1,932 room minus 256), ran no tools, starts after a blank line, and was cut off with '[This note was cut off at the 1,676-token output limit before it finished.]'. Sessions 2-4 skipped it because it could not fit (used 50,162, 48,607 and 40,097 of 40,000), and their handoffs mark the results as unseen. Stalls were 0, 1, 2, 0, correct because sessions 2 and 3 changed no file. Progress check text is correct and there is no sub-agent request below 3. Original request is kept in every <request>; binding rule leads every continuation. Pairs match. Model behaviour: it read the 967-line queryEngine.ts in parallel with other files and re-read it in each session, which fills the 40K window and stops convergence; handoff writers honestly recorded trimmed gaps. Cost recomputes exactly ($0.036087675, 26 requests). Review note: my helper scripts were in the session scratchpad, not a run folder; I wrote nothing into run folders, and ran npm test only in the c7 and j6 projects."
  },
  {
   "id": "c5-huge-log",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "REPORT.md is correct against my own grep/sort/uniq: W20 69,326, W10 46,455, E404 18,730, E500 11,674, E200 6,908; E500 first 2026-10-25T00:00:15, last 2026-10-25T20:26:37.",
   "notes": "The model used Bash aggregation and never read the 12.5 MB file, so FileRead truncation and large-output paging were not exercised here. Context stayed at 14,596 of 1,000,000. Cache read ratio 95%. Cost recomputes exactly ($0.00142586, 4 requests)."
  },
  {
   "id": "c6-many-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "notes/INDEX.md lists all 150 notes exactly once under the correct topic (checked by script: none missing, duplicated or misgrouped); 152 tool calls all paired.",
   "notes": "Context reached 42,219 of 120,000, so no context-full or compaction occurred. The model wrote INDEX.md inside notes/ rather than the project root, generated by a Bash script after reading all 150 files in parallel batches; its final reply states the path. Cache read ratio 88%. Cost recomputes exactly ($0.01498574, 10 requests)."
  },
  {
   "id": "c7-multi-turn-cache",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 4 turns completed: JSDoc added and count() added with a test. My npm test gives 6 tests, 1 pass, 5 fail, matching the model's report; it ran the suite before changing code, found all 5 existing tests already failing, and left them alone.",
   "notes": "Cache looks healthy: 202,863 of 213,350 input tokens read from cache (95%), with only 10,455 written over 12 requests, so the prefix is not being invalidated. Limit of this check: logs and sessions hold only totals and the last request, so 'cacheReadTokens > 0 from the second request' and the per-turn rise could not be confirmed request by request. learned/patterns.json agrees with the commands: 'npm test' runs 3, successes 0 (every run piped, status unknown); the node --test name-pattern check runs 1, successes 0. No pattern is trusted. Cost recomputes exactly ($0.005966705, 12 requests)."
  },
  {
   "id": "j6-long-session",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 6 turns completed in one session: validation, src/cli.js, README and fixes. My npm test gives 37 tests, 37 pass, and the final summary accurately recalls earlier work.",
   "notes": "Context reached 55,288 of 90,000, so no handoff was needed; that is allowed by the check. One 'Still waiting for the model (20s)' heartbeat appeared correctly. learned/patterns.json shows 'npm test' runs 3, successes 0, consistent with status unknown on piped runs. Cache read ratio 93% (541,591 of 583,817); as in c7, only totals are saved. Pairs match and the session file is mode 600. Cost recomputes exactly ($0.02626401, 17 requests)."
  }
 ]
}