{
 "model": "claude-haiku-5-5",
 "category": "edit",
 "scenarios": [
  {
   "id": "b1-inventory-fix",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 5 bugs fixed and exportCsv added. npm test on a scratch copy gives 9/9 and the original test file is unchanged. Exit 0, 1 turn-complete, 13 tool_use/tool_result pairs matched in frames and in the saved wire messages.",
   "notes": "Baselines of src/inventory.js and src/report.js match the seed bytes. test/report.test.js has an absent baseline. No duplicate outputs, and the last outputs frame equals the saved outputs. Cost recomputes to $0.005184. Learned pattern 'npm test' has runs 3, successes 0, which is correct because each command ended in '; echo' or '| tail'. No leftovers, temp files or locks."
  },
  {
   "id": "b2-inventory-approvals",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "5/5 tests pass on a scratch copy with tests unchanged. Five approvals (3 Bash, 2 FileEdit), each with its own requestId, asked once and each followed by its run. FileReads asked for nothing.",
   "notes": "All 8 calls settled complete. No identical call was asked twice and nothing hung. Baselines equal the seed. Cost recomputes to $0.002843. The backslash-escaped cd into the spaced project path was still recorded as 'npm test' (runs 2, successes 0, status unknown after '| tail'), as designed."
  },
  {
   "id": "b3-inventory-deny",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "The single Bash approval was declined. Its state is 'denied', the frame error reads 'You declined this Bash call. Nothing was run.', and the wire tool_result says 'Not run: the user declined approval…'. The project is identical to the seed (diff -rq empty) and its tests still fail 5/5 as seeded. Turn 2 answered accurately.",
   "notes": "Approvals 1, with no re-ask or workaround by the model. No outputs and an empty journal. Two turn-completes, exit 0. The denied call has a tool-result frame with no tool-use frame, as designed, and is paired in the saved session. Cost recomputes to $0.001035."
  },
  {
   "id": "b4-js-to-ts",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Converted to strict TypeScript. On a scratch copy npx tsc --noEmit exits 0 and npm test passes 6/6, including the empty-array case. Old .js files are gone.",
   "notes": "Outputs mark src/math.js and src/stats.js deleted:true with baselines equal to the seeded bytes. package.json baseline equals the seed, and package-lock.json has an absent baseline with a Bash version. 3 failed calls, all genuine: one Bash exited 1 from a trailing 'cat .gitignore', and two FileWrites hit 'exists but has not been read' after the model's mv. Those two are the intended read-first check inside call(), so 'failed' is correct. Guard did not fire. Cost recomputes to $0.009531. tmpMoved t.ts is the model's own file."
  },
  {
   "id": "b5-python-lib",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "unittest discover gives 3/3 OK on a scratch copy. shop/cart.py is fixed and tests/test_cart.py is byte-identical.",
   "notes": "The model added an empty tests/__init__.py, needed for discover -t ., and disclosed it. Baseline of cart.py equals the seed; __init__.py has an absent baseline with a zero-byte version. Cost recomputes to $0.002464."
  },
  {
   "id": "b6-rename-across-files",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "getUserName has 0 references left and getDisplayName has 31. The model's check-imports.mjs reports 16 modules, 0 failed on a scratch copy.",
   "notes": "Outputs hold 17 entries with no duplicates: 16 src files plus the script. The 15 mod baselines equal regenerated seed bytes, and user.js has its baseline with one version. One failed Bash was genuine: BSD sed does not know \\b, so the final grep for the new name exited 1 and the model redid the rename with perl. The no-op sed -i created no spurious extra version. Cost recomputes to $0.003720."
  },
  {
   "id": "b7-notebook",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Used NotebookEdit twice in place. sales.ipynb is valid JSON, nbformat 4.5, 4 cells in the requested order, and the new code cell has outputs and execution_count.",
   "notes": "Baseline is the seeded 242-byte file; two versions follow (638 and 1060 bytes), snapshots hash-verified. NotebookEdit added ids to the two seed cells, which is required at nbformat 4.5 and is what the tool documents. The model's nbformat venv lives in tmp-left. Cost recomputes to $0.003150."
  },
  {
   "id": "b8-large-file-edit",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Line diff of baseline against the result shows only the requested changes: JSDoc added above fn0, fn42 removed, fn100/fn1500/fn2999 multiply. Importing on a scratch copy gives 300, 4500, 8997, fn42 undefined and 2999 exports.",
   "notes": "Baseline equals a regeneration of the seeded 3000-line file. One FileEdit with 5 edits. The model used Bash and Grep with no FileRead, so FileRead pagination was not exercised here. FileEdit without a prior read is allowed because the stale check applies only when a read state exists. Cost recomputes to $0.004926."
  },
  {
   "id": "j3-refactor-legacy",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "InquiryForm, LotArt and Header were split (3 files changed, 16 new, none removed). A fresh esbuild bundle of the copy succeeds, with --conditions=style required before and after alike. The model's DOM captures in tmp-left match before versus after in 22/22 files once opacity and transform are masked. Its tsc outputs are empty before and after.",
   "notes": "4 failures among 77 calls, guard did not fire (no guard text, each followed by a success). All four ran and settled 'failed'. FileEdit on Subjects.tsx was told 'modified externally' after the model's own sed -i, which is the intended stale check inside call(); the model re-read and the edit succeeded. Three Bash calls exited 1: a node harness run, a grep with invalid repetition count, and diff. The changed and new files on disk match the outputs exactly: 19 entries, no duplicates, and the three edited originals have baselines equal to the seed. /tmp writes are not listed, by design. Long-prompt pricing applied to 47 of 56 requests and cost recomputes to $0.647604. tmpCollisions are names left by earlier runs and out-8965.css is in tmpMoved, as the harness note expected. The 'Still thinking' heartbeat appeared."
  },
  {
   "id": "j4-docs-from-code",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "docs/ has 11 module pages plus README, every README link resolves, a regex scan found every exported function, class, const, interface, type and enum named in its page, and engine/ is unchanged.",
   "notes": "12 outputs with absent baselines and no duplicates. All 28 calls complete. The words 'Stopped' and 'fallback' in the log appear only in matched log lines from this task's source and docs work, with no such status kinds or error frames. Cost recomputes to $0.019426. Heartbeat appeared."
  },
  {
   "id": "j5-git-project",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "npm test gives 5/5 on a scratch copy with tests unchanged. Git history is init plus one fix commit touching only src/inventory.js and src/report.js. Working tree is clean.",
   "notes": "Baselines equal the seed and git HEAD~1 content. The commit author G-Code <gcode@localhost> was chosen by the model through git -c because no identity is set in the run HOME, and it disclosed this. One failed Bash was genuine: 'git config user.email' exited 1 with no identity. Cost recomputes to $0.004886."
  }
 ]
}