{
 "model": "claude-haiku-5-5",
 "category": "build",
 "scenarios": [
  {
   "id": "a1-static-site",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Five-page bakery site built. I reran node --test in a scratch copy: 19/19 pass. All 53 local href/src references across the 5 pages resolve. All 3 BrowserVerify reports read passed with 0 errors. 35 calls in one turn, all complete, with no step limit hit.",
   "notes": "Exit 0, turn-complete, .err empty, 35/35 tool_use paired in the saved session and frames. Background python server: tasks frames go running to stopped (143) through TaskStop; leftovers empty and port 8700 free now. All 39 outputs have baseline absent:true and hashes match the files on disk; gallery-lightbox.json holds 2 versions. Cost recompute matches result.json exactly at $0.044828815, cache read ratio 0.915. 5 heartbeats from 20s to 1m20s, all before output, and they stop once output arrives. Learned pattern 'node --test' has runs 1, successes 0 after '| tail', which is correct. The model deleted its own gallery plan with rm and re-planned because of a bad step of its own; BrowserPlan never refused it. Good-plan approval is not exercised because the run is in bypass (approvals 0). Reviewer housekeeping: I left review-scratch/ in all 10 run folders. One rm of my own scratch was refused by Claude Code's safety check, so I did not retry and the folders remain for the user to remove. In a2, a3 and a6 they include copied node_modules. They did not touch project/ or home/."
  },
  {
   "id": "a2-vite-react",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Vite React TS app scaffolded and installed; heading changed to 'Hello from G-Code'. Curl of /src/App.tsx in the log shows the new text. I reran npm run build in a scratch copy: rc 0.",
   "notes": "The tasks frame shows the dev server running at turn end. background-tasks.jsonl records stopped, exit code 143, endedWithProcess true. Leftovers empty, but the runner measures them 5 s after the driver exits (runner.ts:145), so the ~3 s figure in the check text was not timed. App.tsx versions: v0 is the scaffold, v1 has the new heading. Baseline absent:true is right because the project was empty. Cost matches at $0.00369393, ratio 0.961. The model wrote a curl body to /tmp itself; that is model choice. No heartbeats."
  },
  {
   "id": "a3-vite-build-preview",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Vanilla-ts Vite app with a three-origin coffee section. I reran npm run build in a scratch copy: rc 0. The log shows curl 200 for index.html and the JS asset. Server stopped with TaskStop (143), and the model confirmed port 8710 closed.",
   "notes": "The check text expects the baseline to hold the scaffold content, but that does not match this flow. The scaffold was created by the agent's own Bash in an empty project, so baseline absent:true is the true session start. The scaffold content is kept: main.ts versions [2464, 3277] and style.css [5060, 5812], with v0 being the scaffold without the coffee text. I treat this as stale check text, not a defect (fairly sure). dist/ and node_modules are not listed in outputs. Learned pattern 'npm run build' has runs 1, successes 0 after '; echo', which is correct. Cost matches at $0.005331755, ratio 0.933."
  },
  {
   "id": "a4-canvas-game",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Snake game built (index.html, game.js, logic.js). I reran npm test in a scratch copy: 66/66 pass; node --check game.js OK. The BrowserVerify report passed with 22 steps and 0 errors. The failure guard did not stop the test-edit loop.",
   "notes": "The model really sent invalid JSON: 9 FileEdit calls in one reply, each with \"old_string\">. This is a model error, as all 9 show the same malformed key. All 9 settled refused with is_error true and paired in the saved session. New first line checked on all 9 against the saved inputPreview: character 154 is the '>' (1-based), M equals inputChars and the preview length, the before and after fragments match, and Python's decoder gives the same position. The second line's INVALID_JSON equals the preview, under the 2000 cap. Invalid calls have no tool-result frame by design, so toolStates shows refused 9 while toolFailures is 0. The model redid them as one edits call, which applied. The one 'gpci' substring hit is inside base64 image data, not GPCI. 5 heartbeats, all stopping at output. Learned pattern 'npm test' has runs 2, successes 0 after pipes, which is correct. Cost matches at $0.04629001, ratio 0.919."
  },
  {
   "id": "a5-md-docs",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Zero-dependency build.mjs with tests. I reran npm test in a scratch copy: 51/51 pass. npm run build produced all 6 site/*.html pages, with a nav and title checked in api.html.",
   "notes": "Outputs list has 9 entries (package.json, build.mjs, test file, 6 site pages), complete and hash-matched; the pre-existing content/ is correctly not listed. Harness issue, not G-Code: the seed's backticks were eaten by the shell, so content files hold duplicated text and no fenced code. Longest heartbeat series ran 20s to 2m35s, spaced about 15 s, then stopped 12.5 s before the Thinking and Generating statuses. Learned patterns: 'npm test' runs 2, successes 0 (pipes); 'npm run build' runs 1, successes 1 (&& form). The later 'find ... && npm test' is correctly not recorded. scenarios.json says medium effort, but the config and session used high, as the brief states. Cost matches at $0.043628975, ratio 0.905."
  },
  {
   "id": "a6-express-api",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Express notes API with static page. The log shows create, list, update, delete and 404 exercised. I probed a scratch copy on port 18725: list 200, create 201, delete of a missing id 404, static 200. I stopped it and confirmed no listener remains.",
   "notes": "Background server stopped through TaskStop (143); TaskOutput worked while it ran. Leftovers empty and port 8725 free. Outputs list 4 files with node_modules excluded. No learned patterns, as expected with no check commands. Cost matches at $0.00540342, ratio 0.945."
  },
  {
   "id": "a7-chinese-site",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Four-page Chinese site with menu.json and check.mjs. I reran node check.mjs in a scratch copy: 4/4 pass, rc 0. All 26 local references resolve. The BrowserVerify report passed. The final summary is in Chinese.",
   "notes": "No U+FFFD or mojibake pattern in project files, log.jsonl or the saved session; CJK text is intact end to end. The one failed Bash is the model's deliberate negative test, exit 1, correctly settled failed with a single 'Error:' prefix and intact Chinese output. The model deleted its first plan with rm; the outputs entry is marked deleted:true with its version kept, which is correct. BrowserPlan was not refused. Cost matches at $0.014790595, ratio 0.934. No heartbeats."
  },
  {
   "id": "j1-todo-fullstack",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Full-stack todo app built. I reran npm test in a scratch copy: 45/45 pass. Background server was curled and stopped, port 8955 closed. The BrowserVerify report passed with 32 steps and 0 errors.",
   "notes": "Round 19 BrowserPlan fix exercised live. The missing entry index.html settled refused (never failed or denied), with 'Not run: BrowserPlan call refused: Entry must be a project HTML file; ... which does not exist. Nothing was executed.' Nothing was saved: the corrected call (entry public/index.html) saved to the same path without an already-exists refusal. Good-plan approval is not exercised because the run is in bypass. By design the refused call has a tool-result frame and no tool-use frame (23 versus 22); it is paired in the saved session. TaskStop reported exit 0 because the app handles SIGTERM itself, and the state reads stopped. The BrowserVerify dev server left no leftover. 3 heartbeats stopping at output. Learned pattern 'npm test' has runs 1, successes 0; the 'rm -rf data && ... npm test' call is correctly not recorded. Cost matches at $0.033104385, ratio 0.926."
  },
  {
   "id": "j2-cli-tool",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "csvstat.mjs with generator script and tests. I reran npm test in a scratch copy: 30/30 pass. node csvstat.mjs sample.csv reports 2000 data rows, 11 columns, rc 0, matching the model's summary table.",
   "notes": "sample.csv has 2098 physical lines for 2000 data rows, which fits quoted newlines in the notes column (the model's Python csv cross-check gave 2000 rows, 0 ragged). The one failed Bash is the first probe: `cat package.json` in an empty folder made the command exit 1, correctly settled failed. Learned pattern 'npm test' has runs 1, successes 0; 'npm run sample && ...' is correctly not recorded. 2 heartbeats stopping at output. Cost matches at $0.016704345, ratio 0.922."
  },
  {
   "id": "j7-sonnet-sanity",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "TABLE.md with 8 planets and verify-table.mjs. I reran the script in a scratch copy: PASS, 8 rows, rc 0.",
   "notes": "Small baseline: 6 requests, 6 tool calls, all complete, $0.002888065 matching the recompute, ratio 0.952. The first verify run printed FAIL from a regex bug of the model's own, but the command ended in `; echo`, so exit 0 and complete is correct. The model fixed it and reran. Request tools number 31, Skill catalog has 6 skills, no DSH tool, no DeferredToolCall. Moon counts are the model's unverified figures, as it said."
  }
 ]
}