{
 "model": "claude-haiku-5-5",
 "category": "context",
 "scenarios": [
  {
   "id": "c1-handoff",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Three sessions with 2 handoffs; ARCHITECTURE.md finished (11 modules, data flow, 3 risks). I grepped all 80 backticked names against engine/ and none were missing; line counts in seed 1 match wc -l.",
   "notes": "Exit codes [0,0,0], .err empty, no leftovers. 38/38 tool pairs match in the log; every saved session's wire is paired; cost matches recompute12. Read-before-handoff was sent both times with no tools run and no refused tool states. Neither note was cut off. Each saved interrupted message puts the note after a blank line (session 1: 'in full.\\n\\n## Handoff'; session 2: 'before writing.\\n\\nThe context window is nearly full'). Neither handoff file has an unseen-results heading. Stalls were correct: frames carry 1 then 2; continuation 2 says 1 session and continuation 3 says 2 sessions, with no sub-agent request. The binding rule leads both continuations and the original request is kept in <request>. Old sessions are marked interrupted and linked by id and path. Tools 31, Skill catalog 6 lines, no MCP, DSH, auth or learned-pattern effects. Model behaviour, seeds accurate: session 2 spot-checked some finished files, and re-read queryEngine 515-967 after its own first read was cleared by the by-design old-result clearing (45,259 → 32,795). Session 2's handoff also repeats a progress remark in the model's own words; the engine count does not read it."
  },
  {
   "id": "c2-compact",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "One turn with one context-full question and 3 compactions, plus 2 old-result clearings; no API errors. ARCHITECTURE.md covers all 11 modules, data flow and 3 risks; all 77 quoted names exist in the source.",
   "notes": "Exit 0, 57/57 tool pairs, saved wire paired, cost matches recompute12. The original request stays pinned as the first message next to the summary. The '61,585' figure is one parallel read batch going over the simulated 60K limit; compaction ran before the next request. Model behaviour: Haiku re-read queryEngine.ts many times (48 FileReads). Its summary said 'Reads were truncated at ~5,000 chars', although G-Code's trim note says such cuts are in the transcript copy only, not a tool limit. Those trims are G-Code's by design: auto-compaction shortens blocks to fit one chunk, so a summariser never sees the middle of a large read. This cost extra reads but the task finished. Recorded as an observation, not a defect; mid-turn transcripts are not saved, so the 5,000 figure's source is inferred from code."
  },
  {
   "id": "c3-stop-then-continue",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "'stop' ended turn 1 with a clear 92% error, saved as interrupted. /compact shrank 46,878 → 16,984 and turn 3 read only queryEngine 515-967 and finished ARCHITECTURE.md (all 110 quoted names found in source).",
   "notes": "Exit 0; the single error is the designed stop message. compactArchives keeps the original 6 messages. I replayed the archived messages through snapshot summarizeContext with a mock client (no network; scratch script removed). It gave 3 parts, each headed 'Part i of 3' with the user's real request, not the interruption note. Results are labelled '[tool result of FileRead <path>]'; part 2 ends with '[this block continues in the next part]' and part 3 opens with '[continued from the previous part: …hooks.ts]'. Unread results are labelled as not seen yet. Paths here are under 160 characters, so the long-path tail rule was checked only by reading describeCall. The saved summary matches, including hooks.ts lines 187-274 in part 3. Part 1 says no file contents were seen 'here' while pointing to later parts; this caused no harm. Requests add up to 9 (2+3+4). Cost matches recompute12."
  },
  {
   "id": "c4-handoff-chain",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Four sessions with 4 handoffs. SUMMARIES.md grew across sessions 1-3 to 10 unique engine/ entries, no duplicates. queryEngine and engine2 are unfinished because the driver stops after 3 hops under the 40K limit.",
   "notes": "Exit codes all 0, 36/36 tool pairs, no leftovers, cost matches recompute12. All 3 continuations lead with the binding rule and <request> holds the original request. All 4 sessions sent read-before-handoff, ran no tools, and have no unseen heading in the handoff file. Stalls correct: 0 for sessions 1-3, which wrote files, so no Progress check; session 4 changed nothing and its frame carries stalls 1. That fourth handoff is not followed because of the driver's hops<3 limit. Paragraph breaks are correct in all 4 interrupted messages. Session 3's note was cut off with '[This note was cut off at the 2,429-token output limit…]'; 2,429 equals 40000-37315-256, so the cap was correct and the note kept. Model behaviour, seeds accurate: session 3 read 331-967 and session 4 re-read queryEngine from line 1 instead of summarising from the notes ('has to come from the file itself'). That file is too large for one session at this limit, which is why progress stopped. The unfinished task is not caused by G-Code."
  },
  {
   "id": "c5-huge-log",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "REPORT.md is correct: my grep/sort/uniq gave W20 69326, W10 46455, E404 18730, E500 11674, E200 6908; E500 first 00:00:15, last 20:26:37 in file order; it also discloses the clock wrap at line 86,401.",
   "notes": "Exit 0, 5/5 tool pairs, 6 requests, cache ratio 0.948, final context 16,449 of 1,000,000, no leftovers. The model used only small Bash queries (largest output 804 chars), so FileRead's large-file and Bash truncation paths were not exercised here; no claim is made about them from this run. Context did not grow out of control. No FileRead of app.log happened."
  },
  {
   "id": "c6-many-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "INDEX.md lists all 150 notes exactly once under the correct Topic 0-6 (checked by my own parse: 150 unique, 0 duplicates, 0 wrong topics).",
   "notes": "Exit 0, 8/8 tool pairs, 7 requests, cache ratio 0.886, final context 29,314 of 120,000, no compaction needed. The model read 3 files, then one Grep (18,509 chars) and shell loops, rather than 150 small calls; that is a valid choice. Scenario's context-full path not reached."
  },
  {
   "id": "c7-multi-turn-cache",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Four turns done: explanation, JSDoc on the methods, count() with a test, and an npm test report. My npm test gives 6 tests, 1 pass, 5 fail, exactly as the model reported; the 5 are failures in existing tests that the prompts never asked it to fix.",
   "notes": "Cache: read 197,244 of 207,449 input tokens (0.951), writes only 10,173 over 12 requests, no cacheInvalidations. Per-request usage is not saved; total writes are too small for request 2 or later to have missed the prefix, so reads from request 2 are inferred. Learned patterns correct: key 'npm test', runs 3, successes 0, broken 0, since all 3 commands were piped (status unknown); one session listed. Exit 0, 11/11 tool pairs, cost matches recompute12."
  },
  {
   "id": "j6-long-session",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 6 turns completed in one session with no handoff (63,050 of 90,000 used). Validation, CLI, README and fixes done; my npm test gives 43/43 pass, and the final summary correctly recalls the whole day's work.",
   "notes": "Cache ratio 0.958, no invalidations, persistence intact: 12 messages, 60 context messages, wire paired. The one Bash failure is the model's unquoted $OLDPWD on a path with a space, and it recovered. A FileEdit with malformed JSON from the model was correctly marked Not run, with no tool-use frame and a paired saved result; toolHistory holds 35 calls against 34 frames for this reason, and the model retried. Learned patterns correct: 'npm test', runs 6, successes 0, broken 0, all piped; the non-check shell calls were not recorded. Exit 0, no leftovers, cost matches recompute12."
  }
 ]
}