{
 "model": "claude-haiku-5-5",
 "category": "context",
 "scenarios": [
  {
   "id": "c1-handoff",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "4 sessions and 4 handoffs (the driver follows at most 3). ARCHITECTURE.md (10,359 bytes, 11 modules) was written in session 3; session 4 was still fixing draft claims when the driver cap ended the chain, so the unfinished last step is the harness cap. Checked frames, seeds, saved sessions and the .md handoffs.",
   "notes": "Read-before-handoff ran on all 4 handoffs, with no tool calls and no refused states. All 4 replies start after a blank line in the saved interrupted message, and none was cut off (largest reply about 6,761 output tokens). The writer treated the results as read: none of the 4 .md seeds has an unseen marker. Input plus cap stayed inside the 60K window. Stalls 1, 2, none, 1 match the files actually changed. Progress check reads 1 then 2 and is absent after session 3, which wrote the file; no sub-agent wording, since 3 was never reached. Binding rule leads every continuation and <request> keeps the original request. Seeds were accurate: queryEngine 515-967 really was unread because FileRead stopped at line 514. Old sessions are saved and linked, marked interrupted with the handoff path. Pairs intact, exit codes 0, no leftovers, cost matches recompute at $0.06313591. Model behaviour: session 4 spent its window re-verifying the draft; session 2 re-read queryEngine.ts from line 1 although the seed recorded lines 1-197 and 460-514 as read."
  },
  {
   "id": "c2-compact",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Context-full asked once; 5 mid-turn compactions and 2 old-result clearings, no API errors. ARCHITECTURE.md (21 KB) is complete with modules, data flow and three risks.",
   "notes": "Clearings ended at 57.8% before the output reserve, then the turn went on. The saved context keeps the user's request verbatim as message 0 and a labelled summary next to it. Pairs intact, exit 0, cost matches at $0.069500265. Model behaviour: 46 FileReads over 11 files, re-reading after compaction in order to verify, despite summaries saying to rely on them."
  },
  {
   "id": "c3-stop-then-continue",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Stop gave a clear error at 93% (55,853 of 60,000 including reserve) and the turn was saved as interrupted. /compact went 46,932 to 17,286 tokens and the continuation wrote ARCHITECTURE.md (17 KB), reading only queryEngine.ts from 515. No 400s.",
   "notes": "Saved compacted text is 'Summary of Part 1 (of 3)' through Part 3 with the user goal in each. I replayed summarizeContext on the saved compactArchives with a stub client (scratch HOME): 3 requests, each 'Part i of 3' with the user's request, not the interruption note. Results are labelled '[tool result of FileRead .../engine/hooks.ts]'. Part 2 ends with '[this block continues in the next part]' at hooks.ts line 186 and part 3 opens with 'continued from the previous part'. The saved summary matches that cut. No path in this run was long enough to need truncation, so long-path labels were not exercised live; describeCall keeps the path's tail by code. One failed Grep was the model guessing a missing memory/ folder, reported correctly. Exit 0, cost matches at $0.02046587."
  },
  {
   "id": "c4-handoff-chain",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "4 sessions and 4 handoffs. SUMMARIES.md has 6 of 22 files, no duplicates, 'grows across sessions' holds. The chain ended at the driver's follow-up cap with engine/queryEngine.ts (52 KB) still unsummarised inside a 40K window, so the unfinished task is not a G-Code fault.",
   "notes": "Session 1 sent the read-before-handoff request (estimated 30,010, real 30,608); input plus cap at most about 38,842 of 40,000, so no overrun. Reply started after a blank line, no tools, not cut. Sessions 2-4 were at 43,179, 39,737 and 43,117 of 40,000, so the read was correctly skipped for lack of room, which the brief says is not a defect. Their results went to the writer as unseen, and session 2's seed says so explicitly. Stalls none, none, 1, 2 match the files changed; progress paragraph is absent, absent, then 'Progress check: 1 session'. Binding rule leads every continuation; every chained <request> is the ORIGINAL request. Handoff frames' own request field carries the full previous continuation; the next continuation still extracts the original. Model behaviour: Haiku re-reads queryEngine.ts in parallel batches that overflow the window, so sessions 2-4 stalled on it. Failed Bash was 'ls SUMMARIES.md' before the file existed. Exit 0, cost matches at $0.036141755."
  },
  {
   "id": "c5-huge-log",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "REPORT.md is correct: W20 69,326, W10 46,455, E404 18,730, E500 11,674, E200 6,908; E500 first 00:00:15, last 20:26:37. Verified with grep, sort and uniq on the 12.5 MB log.",
   "notes": "The model used 4 Bash aggregations and never read the log with FileRead, so FileRead's large-file path and persistence of big Bash output were not exercised here. Context stayed at 16,590 of 1M. Cache ratio 0.947, exit 0, cost matches at $0.00286801."
  },
  {
   "id": "c6-many-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "INDEX.md lists all 150 notes exactly once under the right Topic (mod 7), checked by script.",
   "notes": "7 tool calls in total (Glob, 4 FileRead, Grep, Bash); no context-full was reached at 120K, so that path was not exercised. Cache ratio 0.857, exit 0, cost matches at $0.006029015."
  },
  {
   "id": "c7-multi-turn-cache",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "4 turns complete: explanation, JSDoc on every method, count() with a passing test, npm test reported honestly. Re-ran npm test on a scratch copy: 1 pass, 5 fail, the failures being the seed's existing bugs.",
   "notes": "Cache read 162,457 of 175,254 input (0.927) over 10 requests, all reporting cache; only totals are saved, so per-request ratios cannot be shown, but cache writes total only 12,769, so later turns read the prefix. No sign of a changing system prompt invalidating the cache. The 2 'failed' Bash states are the npm test exits, correct. Exit 0, cost matches at $0.005609495."
  },
  {
   "id": "j6-long-session",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "6 turns complete, no handoff needed under 90K (59,754 used). Validation, src/cli.js, tests and README done; the final summary knows the earlier work. Re-ran npm test on a scratch copy: 40/40 pass.",
   "notes": "4 FileEdit calls in one reply had invalid JSON (the model wrote \"old_string\"> ...). G-Code settled them refused / Not run, nothing executed; the result names character 153 and the text there, matching the saved inputPreview, and the model retried correctly. Single-model slip, not a defect. No repeat notes, no temp or lock files left. Cache ratio 0.959, exit 0, cost matches at $0.03611598."
  }
 ]
}