{
 "model": "claude-haiku-5-5",
 "category": "edit",
 "scenarios": [
  {
   "id": "b1-inventory-fix",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 5 seeded bugs fixed and exportCsv added with tests. I reran npm test: 11/11 pass, and test/inventory.test.js is identical to the seed.",
   "notes": "Exit 0, turn-complete, 12/12 tool pairs, empty leftovers, empty driver.err. Baselines match the seed hashes (inventory.js 300ad1e8, report.js 9b1a8507), and report.test.js is marked absent. report.js keeps both versions, and the last version of each file matches disk. Learned npm test has runs 3, successes 0, which is correct because each command ended in `; echo` or `| tail`. Cost recompute diff 0. My reruns used the real HOME, so npm may have written debug logs to ~/.npm/_logs outside the run folder."
  },
  {
   "id": "b2-inventory-approvals",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Bugs fixed under default mode with tests unchanged. I reran npm test: 5/5 pass.",
   "notes": "5 approvals (3 Bash, 2 FileEdit), each asked once and followed by its tool-use and result. FileRead asked for none, and nothing hung or was re-asked. Baselines match the seed. Learned npm test has runs 2, successes 0, correct for `| tail`. The model's first call was `cd . && ls`, which is not a check and was not recorded. Permission stayed default."
  },
  {
   "id": "b3-inventory-deny",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "The single Bash call was denied and did not run. The project is byte-identical to the seed (diff -r), and the second turn answered normally.",
   "notes": "State is denied. The frame reads 'You declined this Bash call. Nothing was run.', and the wire result starts 'Not run: the user declined approval…' with is_error set. The model did not retry, so there was 1 approval and no re-ask. The result frame with no tool-use frame is by design, and the saved session pairs 1/1. No outputs and no learned patterns. The reply 'declined when it ran' is loose model wording."
  },
  {
   "id": "b4-js-to-ts",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Converted to strict TypeScript with an empty-array RangeError. I reran `npx tsc --noEmit` (exit 0) and npm test (5/5 pass).",
   "notes": "Changes data is right: math.js and stats.js are marked deleted with seed baselines (09df7dcb, 53e522bc), and the .ts files have absent baselines with the moved bytes as version 1. package.json has baseline ff6a8814 and 3 versions, and package-lock is recorded with no node_modules entries. The 2 FileWrite failures were the read-first error after the model's own `mv`. They returned from call() as failed, nothing was written, and the model read and retried. Learned `npx tsc --noEmit` has runs 3, successes 0, correct for `; echo`."
  },
  {
   "id": "b5-python-lib",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "cart.py fixed. I reran the exact unittest command: 3 tests OK.",
   "notes": "Python 3.9 discovery with `-t .` failed until the model added an empty tests/__init__.py. This is the model's choice under 'do not edit the tests'; test_cart.py was not edited. cart.py baseline 4585344c matches the seed, and __init__.py is marked absent. Learned patterns have the two expected keys (with and without -v), each with successes 0, which is correct because each command ended in `; echo` or `| tail`."
  },
  {
   "id": "b6-rename-across-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Renamed in all 16 files. I reran check-imports.mjs (PASS, 16 modules, 15 callers), and grep finds 0 uses of the old name.",
   "notes": "The rename ran through Bash sed. Outputs list 16 Bash entries plus check-imports.mjs with no duplicates. All 16 baselines match recomputed seed bytes, and snapshots and last versions match. The model left a backup in /tmp. Effort was High per the controls frame, despite medium in scenarios.json."
  },
  {
   "id": "b7-notebook",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Code and markdown cells inserted in order with NotebookEdit. The file is valid JSON with nbformat 4.5 and 4 cells, and the model's nbformat 5.10.4 validation passed.",
   "notes": "Baseline 33f5e001 matches the seed and both versions are kept. Cell ids were added to the two original cells, whose source is unchanged. The new cell's output is Mean 7.25, Max 13. The model installed nbformat with pip for the check."
  },
  {
   "id": "b8-large-file-edit",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "My line diff against the recomputed seed shows only the JSDoc, the fn42 deletion and fn100, fn1500 and fn2999 changed. verify-big.mjs passes and my import check gives 2999 exports with correct behaviour.",
   "notes": "Baseline 02247b81 is the original 170,780 bytes, with 2 versions kept. The model used Bash and Grep, so FileRead pagination was not exercised. FileEdit worked with no prior FileRead because its staleness check applies only once a file has been read; this is by design and old_string must still match."
  },
  {
   "id": "j3-refactor-legacy",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "InquiryForm, LotArt and HeroLandscape were split into 12 new files. My before and after esbuild bundles both exit 0 and my import/export check finds 0 problems. The model's rendered HTML and form-flow snapshots are byte-identical before and after, and its before/after source copies match the seed and current src.",
   "notes": "6 failures: 4 Bash and 2 FileEdit stale-file errors after the model's own sed. All ran and settled as failed, 0 as refused. They were never consecutive, so the failure guard did not fire. 60/60 pairs. Changes match disk exactly: 16 entries, no duplicates, baselines equal the seed, and Bash plus FileEdit versions are kept for Contact.tsx and Subjects.tsx. Cost $0.38035 matches the recompute, with 29 long-prompt requests. A flat-card sum differs by $0.27417, which the long-prompt card accounts for. The model chose the largest components by function size, not file size, and built its deps under /tmp with NODE_PATH and --conditions=style because the seed has no node_modules; those /tmp folders remain. My bundles reused that dependency folder, added --external:tailwindcss and an empty CSS loader, and wrote only to stdout. No browser check was done. A scratch folder I made in this run folder was removed, and a later bash -c rm was blocked by a safety check and never ran; the final listing shows no scratch left."
  },
  {
   "id": "j4-docs-from-code",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "11 docs and a README written. My script found every exported function and class from all 11 .ts files named in its doc, and all README links resolve. I spot-checked failureBudget.md against its source.",
   "notes": "12 outputs, all with absent baselines, and the engine files are unchanged from the seed. 30/30 pairs, context used about 67K, cache-read ratio 0.87. No project tests apply."
  },
  {
   "id": "j5-git-project",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Fix committed as a4de930 on top of init, with tests unchanged. I reran npm test (5/5 pass); the working tree is clean.",
   "notes": "Baselines equal the blobs in 639b962, and last versions equal HEAD. The commit touches only src/inventory.js and src/report.js. The diff stat was shown before committing. Learned npm test has runs 1, successes 0: the `find … && npm test` call is correctly not recorded, and `| tail` leaves the status unknown."
  }
 ]
}