{
 "model": "claude-haiku-5-5",
 "category": "edit",
 "scenarios": [
  {
   "id": "b1-inventory-fix",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fixed the bugs and added exportCsv with 4 tests. I ran npm test: 9/9 pass. The seeded test file is unchanged.",
   "notes": "Exit 0, driver.err empty, 1/1 turn-complete, 12/12 tool pairs in frames and context. Baselines for inventory.js and report.js match the seed hashes. report.js has 2 versions, report.test.js has an absent baseline, no duplicates. Cost recompute $0.00545031 matches. Leftovers empty. patterns.json has npm test runs=3, successes=0, correct because each run was followed by `; echo` (status unknown)."
  },
  {
   "id": "b2-inventory-approvals",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fixed the bugs in src/ with the tests unchanged. I ran npm test: 5/5 pass.",
   "notes": "5 approvals (3 Bash, 2 FileEdit), each followed by the tool running within milliseconds. None hung or was asked twice, and FileRead needed none. Baselines match the seed. Cost $0.002737765 matches. patterns.json has npm test runs=2, successes=0: the backslash-escaped cd was read as inside the project and `| tail` makes status unknown; the first Bash call has no check. The log shows the approval frames, not the requestId replies."
  },
  {
   "id": "b3-inventory-deny",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "The single Bash call was denied and never ran. `diff -r` against the seed shows the project identical. Turn 2 explained the denial accurately.",
   "notes": "toolHistory state is denied and the frame reads 'You declined this Bash call. Nothing was run.' The saved tool_result says the user declined approval. One approval only, no re-ask and no workaround by the model. Both turns ended with turn-complete. By design there is a tool-result frame with no tool-use frame, and the saved session pairs them. No outputs key, no patterns file and no .gcode folder. Cost $0.00109158 matches."
  },
  {
   "id": "b4-js-to-ts",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Converted to strict TypeScript with an empty-array error. I ran `npx tsc --noEmit` (exit 0) and npm test (6/6 pass).",
   "notes": "Changes data: math.js and stats.js are marked deleted with seed baselines; math.ts and stats.ts have absent baselines and 2 versions; package.json has the seed baseline and 2 versions; no duplicates. The 2 FileWrite failures were the read-before-overwrite check after a Bash mv. They settled as failed, were followed by FileRead, and then succeeded. Model errors, handled correctly: it called Config with key `scripts`, which wrote project .gcode/settings.json, then removed that folder itself and said so in its final message. Project .gcode stayed readable through Bash as D17-7 intends. The model also left a disclosed copy at /tmp/b4-js-backup. patterns.json has `npx tsc --noEmit` runs=1, successes=0, correct; the --showConfig call was not recorded. Cost $0.007753985 matches."
  },
  {
   "id": "b5-python-lib",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fixed shop/cart.py. I ran `python3 -m unittest discover -s tests -t .`: 3 tests OK. test_cart.py is unchanged.",
   "notes": "The model added an empty tests/__init__.py because discovery needed it, and disclosed this. It is recorded with an absent baseline. The cart.py baseline is byte-identical to the seed. patterns.json has two keys (plain runs=2, -v runs=1), all successes=0 because of `; echo`. Cost $0.003183015 matches."
  },
  {
   "id": "b6-rename-across-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Renamed everywhere: grep finds 0 uses of the old name and 16 files with the new one. I ran check-imports.mjs: PASS.",
   "notes": "Outputs list all 16 edited files plus check-imports.mjs, with no duplicates. All 16 baselines match the seed text. Cost $0.00707701 matches. Scenario effort is listed as medium but the run used High, as the brief says."
  },
  {
   "id": "b7-notebook",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "NotebookEdit inserted a code cell at index 2 and a markdown cell at index 3. The file is valid JSON at nbformat 4.5. I executed the code cells: Mean 7.25, Max 13.",
   "notes": "The existing cells' content is preserved; G-Code added the cell ids that nbformat 4.5 requires. The baseline is the original 2-cell, 242-byte file, with 2 versions (636, 966 bytes), and the last version matches disk. The model validated with nbformat in a /tmp virtualenv and disclosed it. I did not rerun nbformat validation myself. Cost $0.002640585 matches."
  },
  {
   "id": "b8-large-file-edit",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "My line diff against the regenerated seed shows only the requested changes: JSDoc above fn0, fn42 deleted, and fn100, fn1500 and fn2999 changed to multiply.",
   "notes": "The baseline hash equals the original 3000-line seed. FileRead pagination notes were correct, and all edits went in one FileEdit `edits` call. verify-big.mjs is recorded with an absent baseline. Cost $0.00506988 matches."
  },
  {
   "id": "j3-refactor-legacy",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Split Subjects, Contact and Home, which are the three largest component files in the seed. The bundle succeeds with the added externals. I rebuilt both trees and rendered them on the server: all 28 outputs are identical.",
   "notes": "5 failures, all with runningAt set and state failed, which is correct because they ran: 4 Bash (2 background commands that exited at once, 2 foreground non-zero exits) and 1 BrowserPlan with a missing entry. Longest failure streak was 2, with successes in between, and the guard did not fire. No refused calls. 66/66 tool pairs. The three baselines match the seed; 33 outputs, no duplicates. The background-exit notice appeared once. BrowserVerify report: passed, 23 checks and 10 assertions; no listener or serve.mjs process afterwards, and leftovers are empty. The exact requested esbuild command fails on both the seed and the refactored tree with the same kind of error, because the seed's unscoped dependencies are not installed. The model added externals and disclosed this. Files the model wrote under /tmp are not in Changes because they are outside the project, by design (isProjectPath). .gcode/verification/entry.html, written by Bash, is not listed because .gcode is skipped, also by design. Cost $0.209303735 matches a recompute using the long-prompt rate for 15 requests. I ran the render rebuild with the model's /tmp/verify dependencies and entry files; I confirmed /tmp/src-orig-8965 equals the seed. My scratch folder is removed."
  },
  {
   "id": "j4-docs-from-code",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Wrote docs for all 11 engine .ts files plus README.md. My script found every exported function, class and const in its doc page, and all README links resolve.",
   "notes": "engine/ is unchanged from the seed. 12 outputs with absent baselines, no duplicates. No clearing or compaction. Interfaces and types were left out and disclosed, matching the request. I checked export coverage by name, not the wording of each description. Cost $0.0225256 matches."
  },
  {
   "id": "j5-git-project",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fixed the code and made one commit on top of init. I ran npm test: 5/5 pass. The tests are unchanged, the working tree is clean, and git fsck reports nothing.",
   "notes": "Baselines match the seed, and the init commit's inventory.js hash equals the seed's. The commit touches src/inventory.js and src/report.js only. The author is git's default identity from the machine, since the run HOME has no git config. patterns.json has npm test runs=1, successes=0: the `find ... && npm test` call is correctly not recorded and the `| tail` call has unknown status. Cost $0.004031515 matches."
  }
 ]
}