{
 "model": "claude-haiku-5-5",
 "category": "edit",
 "scenarios": [
  {
   "id": "b1-inventory-fix",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 5 bugs fixed and exportCsv added with 7 tests. I reran npm test: 12/12 pass, and test/inventory.test.js is identical to the seed.",
   "notes": "Exit 0, turn-complete, 11/11 tool pairs, no failures, no leftovers. Baselines for inventory.js and report.js match the seeded bytes; report.js has 2 versions and report.test.js has an absent baseline, with no duplicates. Cost recompute $0.005562685 matches exactly. All 3 Bash runs used `; echo` or `| tail`, so patterns.json correctly shows runs 3, successes 0, broken 0. The first run's failing tests still gave the expected fix-and-rerun loop."
  },
  {
   "id": "b2-inventory-approvals",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fixed src/ without touching tests. I reran npm test: 5/5 pass.",
   "notes": "5 approvals (3 Bash, 2 FileEdit), each with a unique requestId and each followed by its own run; none hung or was asked twice. The 3 FileRead calls needed no approval. Baselines match the seed, 8/8 pairs, cost matches. patterns.json shows runs 2, successes 0 (both runs piped to tail), which is correct."
  },
  {
   "id": "b3-inventory-deny",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "The only call was denied and nothing ran. The project is byte-identical to the seed (diff -r clean), and the second turn explained the decline accurately.",
   "notes": "Saved state is 'denied'. The error reads 'You declined this Bash call. Nothing was run.' and the wire result says 'Not run: the user declined approval'. One approval, no re-ask, no retry through another tool, no outputs, no patterns file. The result frame with no tool-use frame is by design, and the saved session pairs 1/1. Both turns ended with turn-complete. The second turn's <context> block carries the session-memory note about the decline, which is accurate."
  },
  {
   "id": "b4-js-to-ts",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Converted to strict TypeScript with an empty-array RangeError and tests. I reran npx tsc --noEmit (exit 0) and npm test (5/5 pass, exit 0).",
   "notes": "Changes data: math.js and stats.js are recorded as deleted (tool Bash, seeded baseline, no versions); math.ts and stats.ts have absent baselines and 2 versions; package.json keeps its seeded baseline with 4 versions; node_modules is not listed. 2 FileWrite failures ('exists but has not been read') came from the tool's own check after the model's mv, settled as failed, and were recovered by reading first; the guard did not fire. Cost matches. Commands whose key exceeds 160 characters are skipped; the rest match patterns.json."
  },
  {
   "id": "b5-python-lib",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fixed cart.py and added an empty tests/__init__.py. I reran the exact unittest command: 3 tests OK, test_cart.py untouched.",
   "notes": "The first run failed with 'Start directory is not importable'. The model disclosed the added __init__.py as the only change outside cart.py. The cart.py baseline matches the seed, __init__.py has an absent baseline with a 0-byte version. Both unittest commands appear as separate keys, each with successes 0 because of `; echo`. Cost matches."
  },
  {
   "id": "b6-rename-across-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Renamed across all 16 files. I reran check-imports.mjs (16/16 import), grep finds 0 old references, and getDisplayName appears in 16 files.",
   "notes": "Outputs has 17 entries (16 sources plus the script) with no duplicates. All 16 source baselines hash to the regenerated seed bytes and all versions match the current files. The first sed attempt changed nothing because BSD sed lacks \\b; G-Code recorded no false change, and the model reported it and redid the rename with perl. No patterns file, which is correct because no check command ran. Cost matches. The scenario lists medium effort but the controls frame shows high (known)."
  },
  {
   "id": "b7-notebook",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Inserted a code cell and a markdown cell in place. The file is valid JSON (nbformat 4.5, 4 cells in the right order), and nbformat.validate passes using the package the model installed under the run HOME.",
   "notes": "The baseline matches the seed (242 bytes) with 2 NotebookEdit versions, and the last version equals the file. NotebookEdit reformats the JSON and adds ids to the existing cells, which nbformat 4.5 expects; the model disclosed this. The model used pip to install nbformat under the run HOME. Cost matches."
  },
  {
   "id": "b8-large-file-edit",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "A diff against the regenerated seed shows only the JSDoc above fn0, fn42 deleted, and fn100, fn1500 and fn2999 changed to multiply. I reran verify-big.mjs and it passes.",
   "notes": "The baseline hash equals the regenerated original 3000-line file (170780 bytes), with 2 FileEdit versions. One FileEdit used the edits array; the second reworded the JSDoc. FileEdit has no read-first rule when old_string matches, only an mtime guard after a read, so the grep-based approach was accepted. The model left /tmp/big.js.orig and verify-big.mjs and disclosed both. Cost matches."
  },
  {
   "id": "j3-refactor-legacy",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Split Subjects, Contact and Home into smaller same-folder components. esbuild bundles both trees with NODE_PATH and --conditions=style. My own render comparison gave identical markup for 27 cases, including all illustrations, Home and Contact.",
   "notes": "2 failures, and the guard did not fire. Bash `sed -i` failed on BSD sed and stayed 'failed' because it ran. FileWrite 'modified externally' stayed 'failed' and was accurate, because the previous Bash call rewrote entry.tsx with node. They were different errors with successes in between. 76/76 pairs. 23 outputs with no duplicates; the 3 changed originals have baselines hashing to the seed and all versions match the files. /tmp harness writes are not in Changes because outputRecord is project-only by design. The stated bundle command cannot run as written on this seed, which has no dependencies; the model disclosed its added flags and left /tmp/gcode-tools-8965 behind. 27 of 42 requests were long-prompt, and the recompute with long pricing is exactly $0.324689615. 3 heartbeat statuses. No patterns file, which is correct because ../node_modules/.bin/tsc and esbuild are not recognised checks."
  },
  {
   "id": "j4-docs-from-code",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Wrote 11 module pages and a README. All links resolve, every exported name I extracted from engine/*.ts appears in its page, and engine/ is unchanged from the seed.",
   "notes": "28/28 pairs, 12 outputs with absent baselines and no duplicates. I sampled failureBudget.md against its source and the signature and defaults match; I did not check every signature. The cache ratio of 0.77 fits the large writes and parallel reads. 1 heartbeat status. Cost matches. The scenario lists medium effort but the controls frame shows high (known). The hooks.ts header remark in the model's report is about fixture text."
  },
  {
   "id": "j5-git-project",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "One new commit 5a81bab on top of init, working tree clean, tests unchanged, git fsck clean. I reran npm test: 5/5 pass. The final message shows the diff stat.",
   "notes": "Baselines for inventory.js and report.js match the seed and the init commit's blob for inventory.js. Versions equal the committed files. patterns.json has `npm test` with runs 2 and `git diff --stat 2624641 HEAD && npm test` with runs 1, all successes 0 because they are piped to tail or grep, which is correct. Commit author is the default machine identity because the run HOME has no git config, which is environment. Cost matches."
  }
 ]
}