{
 "model": "claude-haiku-5-5",
 "category": "context",
 "scenarios": [
  {
   "id": "c1-handoff",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "3 sessions and 2 handoffs; ARCHITECTURE.md (89 lines) was written in session 3 with modules, data flow and three risks; all 93 backticked identifiers exist in engine/*.ts. I checked: exit codes [0,0,0], .err files empty, no leftovers, 28/28 tool-use/result frames paired, all 3 saved sessions are valid JSON with paired contextMessages. The first two sessions are saved with the handed-off turn marked interrupted and the handoff path in it. Both .gcode/handoffs/*.md files open with the binding rule. Cost recomputed from usageTotals is $0.03888552 over 18 requests, the same as result.json; cacheRead is above 0 in every session.",
   "notes": "Read-before-handoff ran at both handoffs (the 'Reading the latest results' status came first and the engine note was on the tool-result message). Neither reply ran a tool or was cut off (10,419 and 3,807 chars; cap = min(96,000 High budget, room-256), about 12.5K at handoff 1). Both start after a blank line ('...in parallel.\\n\\n## Handoff...' and '...consumed.\\n\\nNothing has been written...'). With a reply present, the writer gets unseenFrom = lastReply+1 (session/handoff.ts:153), so the results count as read. Stalls are right: frame 1 has stalls 1, frame 2 has stalls 2 (no file changed in sessions 1-2). Continuation 2 says 'Progress check: 1 session...' and continuation 3 says '2 sessions...', with no sub-agent text (count under 3). The binding rule leads each continuation and <request> holds the original user request both times. Model behaviour: session 2 re-read queryEngine.ts 185-514 and part of hooks.ts that session 1 had already read. The handoff writer's budget-fitted transcript clipped those results and its seed honestly marked those ranges unverified, so the seed is accurate. This is the same kind of re-read as 10h, not a defect. Session 3 did not re-read whole files. The model also asked to confirm the output location but wrote at the project root anyway."
  },
  {
   "id": "c2-compact",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "One turn: context-full was asked once, then 4 mid-turn compactions and 1 old-result clearing (46,218 -> 35,476, under 75%, so no question asked). Every request after a compaction succeeded with no 400 and no error frame. ARCHITECTURE.md (16.5 KB) covers all 11 modules, the data flow and three risks. I checked: exit 0, .err empty, no leftovers, 50/50 frames paired, saved contextMessages paired, and the summary message keeps the user's request verbatim. Cost recomputed is $0.048332965 over 22 requests, matching result.json.",
   "notes": "4 compactions is under MAX_TURN_COMPACTIONS 8, so asking only once is correct. After compaction the kept tool results were clipped by design (clipToolResults, minimum 1,500 chars), so the model re-read files in ranges, using FileRead and later Bash sed. That cost a lot of re-reading but G-Code behaved as designed. 5 cacheInvalidations were recorded, as expected from rewriting history at compaction."
  },
  {
   "id": "c3-stop-then-continue",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Turn 1: 'stop' ended the turn with a clear error ('context window is 92% full... Send /compact') and the turn was saved as interrupted. /compact reduced 46,869 -> 17,522 tokens. Turn 3 read only what was missing (queryEngine.ts 515-744 and 745-967, a 20-line check of the end of hooks.ts) and wrote ARCHITECTURE.md (13.9 KB, all 11 modules). I checked: exit 0, .err empty, no leftovers, 23/23 frames paired, saved session paired, no 400 after /compact. Cost recomputed is $0.020457885 over 15 requests, matching.",
   "notes": "The saved compacted text was split into 'Part 1 (of 3)', 'Part 2 of 3' and 'Part 3'. Each part states the user's real request as the goal (Part 2 says 'latest request'), not the interruption marker. Results are attributed by tool and file path ('engine/hooks.ts, lines 187-274'). The hooks.ts read cut between parts says 'this block continues in the next part', and Part 3 picks it up at line 187. File names survive in all labels. Part 1 says its FileRead results fall in the next part, which is correct for a split between calls and results. Driver turn 1 outcome 'error' is the designed stop, not a crash."
  },
  {
   "id": "c4-handoff-chain",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "4 sessions and 4 handoffs (the driver followed 3). SUMMARIES.md holds 7 of 22 files (engine/ microcompact, hooks, index, compact, context, failureBudget, toolRunner) with no duplicate sections; queryEngine.ts and the rest are unfinished. I checked: exit codes all 0, .err empty, no leftovers, 36/36 frames paired, 4 saved sessions paired with interrupted turns. Each continuation starts with the binding rule and has one <request> with the ORIGINAL request. Cost recomputed is $0.039050175 over 28 requests, matching.",
   "notes": "G-Code did not cause the unfinished task. queryEngine.ts is 52.9 KB in a 40K window, and Haiku insisted on reading it in full before writing its summary. The driver's 3-follow-up cap ended the run. Handoff 2 skipped read-before-handoff correctly: four parallel reads pushed the request to 43,323 of 40,000 tokens, so there was no room (room < 1,024), and the seed said those results had not been seen. Handoffs 1, 3 and 4 sent the read request; the replies ran no tools, were not cut off (4,615 / 4,912 / 7,929 chars) and start after a blank line. Stalls: handoffs 1-2 have stalls 0 (SUMMARIES.md grew), handoff 3 has 1, handoff 4 has 2. Only continuation 4 carries 'Progress check: 1 session...', and none asks for sub-agents (count under 3). Model/design observation: in this small window the handoff writer's transcript is clipped, so seeds list ranges the session did see as 'partial'. The next session then re-reads shifting gaps of queryEngine.ts (sessions 3 and 4 read complementary ranges), which is slow progress but not a loop. One old-result clearing (29,119 -> 23,028) happened in session 1. Worth raising with the user as a design question; not a defect."
  },
  {
   "id": "c5-huge-log",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "REPORT.md is correct: W20 69,326, W10 46,455, E404 18,730, E500 11,674, E200 6,908; first E500 2026-10-25T00:00:15, last 2026-10-25T20:26:37. I confirmed this myself with awk|sort|uniq and grep on the 12.6 MB app.log. Exit 0, .err empty, no leftovers, 5/5 paired. Cost recomputed is $0.002775115 over 6 requests, matching; cache read ratio is 0.95.",
   "notes": "The model used awk/grep in Bash, so every output was small. The large-output persistence path (FileRead or Bash head+tail on the 12 MB file) was not exercised and the context stayed small. It also noted a clock wrap at line 86,401, which is correct about the data."
  },
  {
   "id": "c6-many-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "INDEX.md lists all 150 notes exactly once under the right Topic 0-6 headings; I verified this by parsing it. Exit 0, .err empty, no leftovers, 7/7 paired, no context-full. Cost recomputed is $0.00524157 over 6 requests, matching.",
   "notes": "The model wrote the index to notes/INDEX.md, not the project root; it said so and offered to move it. It read the notes with one Grep '.' over notes/ (450 lines) instead of 150 FileReads, so the many-small-calls and compaction paths were not exercised. Its own duplicate check printed 'DUP/MISSING' because each link has the file name twice; the model read that correctly."
  },
  {
   "id": "c7-multi-turn-cache",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 4 turns completed. The explanation is right, JSDoc was added (6 blocks), and count() plus a test were added. The npm test report (6 tests, 1 pass, 5 pre-existing seed failures) matches my own npm test run. Exit 0, .err empty, no leftovers, 12/12 paired. Cost recomputed is $0.005929995 over 11 requests, matching.",
   "notes": "Cache: 183,537 of 194,596 input tokens were cache reads (0.94), 11,029 were writes and about 30 were uncached, with no cacheInvalidations. The writes are about one prefix, so every request after the first read from cache. The last request read 22,265 of 22,772. Per-request usage is not stored, so the per-turn ratio is inferred from the totals. The model correctly left the failing seed tests unfixed and asked which side is correct. A learned/patterns.json was created as the round-17 feature."
  },
  {
   "id": "j6-long-session",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "7 turns completed: 6 user turns plus the handoff continuation. There was one mid-turn handoff in turn 5, and the continuation fixed the failing tests. My own npm test run: 42/42 pass. The final summary correctly covers the earlier session's work (validation, cli.js, README, tests) taken from the handoff, plus this session's fixes. Exit codes [0,0], .err empty, no leftovers, 38/38 paired, both sessions valid and paired. Cost recomputed is $0.048284975 over 31 requests, matching; cache read ratio is 0.89.",
   "notes": "The handoff sent the read-before-handoff request. The reply (2,929 chars) ran no tools, was not cut off and starts after a blank line. The handoff frame has no stalls (files changed), and the continuation has the binding rule, the original turn-5 request in <request>, and no Progress check. As round 17 expects, the continuation's frozen system prompt has a 'Learned in earlier sessions' section with 3 npm test verification commands from session 1. One of them is 'npm test | tail' recorded as 'worked' even though tests failed; that is by design, since exit 0 counts and failing tests still show the command works. The single tool failure is a grep with no matches (exit 1), correctly shown as failed."
  }
 ]
}