{
 "model": "claude-haiku-5-5",
 "category": "build",
 "scenarios": [
  {
   "id": "a1-static-site",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Five pages, shared css/js, js/validate.js as an ES module and data/menu.json were built. I reran node --test: 21/21 pass. No local href/src is broken. The toggle, lightbox keys and menu fetch were checked only as strings in the code; for behaviour I rely on the agent's BrowserVerify reports (final plans 21/21 and 17/17 assertions).",
   "notes": "35 calls in one turn (33 tool-use frames plus 2 refused), exit 0, turn-complete, .err empty, leftovers empty, wire pairs 35/35. The python server was stopped by TaskStop (exit 143, tasks frame stopped), and port 8700 is free now. Outputs: 38, all baselines absent:true, hashes match disk; only the worker's .gcode/verification/.gitignore is unlisted. Tool states: 2 refused BrowserPlan calls (invalid JSON, then over 60 steps), 1 failed BrowserVerify (the plan expected a trailing period in the title; the real title has none, so the failure is correct). No 'denied'. The driver filters tool-state frames, so states were read from the saved toolHistory. The a1 invalid-JSON cause is not fully verified: only the first 6,000 of 7,632 characters are saved and those have no bracket error, so the fault lies later; the a7 case was confirmed as a model error. Skill browser-verify was loaded once by the model, with no trigger involved. 3 heartbeats, none after output. Learned: `node --test` runs 1, successes 0, correct after `; echo`. Tools 31, Skill catalog 6 lines, no GPCI, MCP, DSH or auth effects, no key-shaped strings. Cost matches recompute12 (max diff 0, no mismatches across all 58). Reviewer process notes: a stray `cat > /tmp/null` in my first command hung it and created an empty file outside the allowed scratch locations; I stopped it and deleted the file. While reading code I also saw grep hits from reviews-haiku-r17d, including its j1 note; no verdict here rests on them. I ran no BrowserVerify myself."
  },
  {
   "id": "a2-vite-react",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Vite React TS was scaffolded and installed, and the heading was changed to 'Hello from G-Code'. The agent's curl of /src/App.tsx showed the new heading. My tsc -b --noEmit printed no errors, and vite build to a temporary folder succeeded (folder removed).",
   "notes": "The dev server was left running at turn end as asked: last tasks frame and TaskOutput both show running. After the driver exited, result.json leftovers are empty and port 8705 is free now; I could not independently time the ~3 s window, so that rests on the runner's check. Exit 0, 10/10 pairs, 19 outputs matching disk. App.tsx has baseline absent:true and 2 versions, correct because Bash scaffolded it in this session. No check command ran, so no learned file. No PATH problem."
  },
  {
   "id": "a3-vite-build-preview",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Vanilla-ts scaffold with a three-origin section. dist/ holds the built JS asset named in index.html. My tsc printed no errors and vite build to a temporary folder succeeded (folder removed).",
   "notes": "The server was stopped by TaskStop (exit 143), the agent's curl afterwards failed to connect, leftovers are empty and the port is free. The scenario check says baselines should hold scaffold content, but the scaffold was created by Bash in this session, so absent:true is right; versions[0] of main.ts (2,464 bytes) and style.css (5,060 bytes) hold the scaffold and the last versions hold the edits. I treat this as stale check text. dist/ and node_modules are not in outputs; 13 outputs match disk. Learned: `npm run build` runs 1, successes 0, correct after `| tail; echo`. 13/13 pairs."
  },
  {
   "id": "a4-canvas-game",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Snake game in index.html, game.js and logic.js. I reran the tests: 93/93 pass. node --check is clean on game.js and logic.js, and index.html loads game.js as a module. The agent's BrowserVerify report passed 12/12 assertions.",
   "notes": "The failure guard did not fire and the turn completed. One BrowserPlan was refused for invalid input (unknown key selector), state refused, with a result frame and no use frame as designed; saved pairs 14/14. 5 heartbeats including one 'Still waiting for the model (20s)', none after output. Learned: `npm test` runs 1, successes 0, correct after `| tail`; the `node --check ... && npm test` call is correctly not recorded. Leftovers empty. The model's report states game.js has no unit tests."
  },
  {
   "id": "a5-md-docs",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Zero-dependency build.mjs with tests. I reran the tests: 51/51 pass. Rerunning the build produced site/ byte-identical to the agent's 6 HTML pages.",
   "notes": "Outputs: 9, complete and matching disk, all baselines absent:true; the seed content files are correctly unlisted. Known harness issue: the printf seed ate the backticks, and the model correctly reported the odd content. driver.config says medium effort but the controls frame says high, matching the brief's statement that effort was High. `node --test test/` failed on Node 26 treating the directory as a file (model/environment), and the model listed the files instead. 11 heartbeats in one long reasoning stretch up to 2m 50s, stopping when output arrived. Learned: `node --test test/` runs 1 and the two-file key runs 2, successes 0, correct after pipes. 11/11 pairs, leftovers empty. Reviewer note: my rebuild rewrote site/ with identical contents, so only mtimes changed."
  },
  {
   "id": "a6-express-api",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Express notes CRUD with a static page. I ran it briefly on port 18725: root 200, POST 201, list 200, PUT 200, DELETE 204, 404 after delete; I then stopped it and confirmed no listener remained.",
   "notes": "The agent's server was stopped by TaskStop (exit 143), the follow-up curl failed to connect, leftovers empty. Outputs: 4 matching disk, node_modules excluded. 9/9 pairs. No check command ran, so no learned file. The model states the page's fetch interaction was not run in a browser."
  },
  {
   "id": "a7-chinese-site",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Four zh-CN pages, one CSS file and menu.json with CNY prices. I reran check.mjs: 4 files, 0 failures. No local link is broken. The final summary is in Chinese.",
   "notes": "0 U+FFFD and 0 mojibake patterns across log and saved session; file contents intact. The refused BrowserPlan (invalid JSON) is a genuine model error: the saved arguments have one extra closing brace, confirmed by parsing them, and the state is refused. Two failed BrowserVerify runs are correct: the text selectors matched 3 and 4 elements (strict mode), with a single 'Error:' prefix, and the third plan passed 10/10 assertions. Saved pairs 24/24; the invalid-JSON call has no tool-result frame in this log, and its refused state was read from toolHistory. Skill loaded once by the model. `node check.mjs` is not a learned check, so no learned file. Leftovers empty, and the BrowserVerify-owned servers are gone."
  },
  {
   "id": "j1-todo-fullstack",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Node http todo app with JSON store and vanilla frontend. I reran npm test: 43/43 pass, data/todos.json unchanged. The agent's curl checks passed and BrowserVerify passed 25/25 assertions.",
   "notes": "The background server was stopped by TaskStop (exit 143); the agent confirmed port 8955 closed and leftovers are empty. BrowserPlan with entry index.html returned 'Error: Entry must be a project HTML file...' and settled failed. This comes from inside call() (BrowserPlanTool.ts, inspectBrowserPlan in browser/validation.ts:16), saved nothing and reads 'Error:', not 'Not run:'. I judge it 'ran and failed stays failed', fairly sure, though it is close to an input refusal, and BrowserVerify's same check in checkPermissions would settle refused; the user may want to decide whether to align them. The message is accurate and the model corrected the path at once. data/todos.json and .gcode/verification/todo-browser-data.json are absent from outputs because the project's own .gitignore covers them. Learned: `npm test` runs 3, successes 0, correct: `cd` to the project dropped, status unknown after pipes. 29/29 pairs."
  },
  {
   "id": "j2-cli-tool",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "csvstat.mjs, gen-sample.mjs and tests. I reran the tests: 30/30 pass. The tool reports 2000 data rows and 9 columns on sample.csv; the file has 2,105 physical lines, and the agent's python csv cross-check also gave 2000 rows.",
   "notes": "No learned file is correct: `node gen-sample.mjs && ... && node --test` has a non-check before the first check, so it is not recorded by design. 2 heartbeats, none after output. 7/7 pairs, 5 outputs match disk, leftovers empty."
  },
  {
   "id": "j7-sonnet-sanity",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "TABLE.md with the 8 planets; I reran verify-table.js: PASS, 8 data rows.",
   "notes": "Small baseline: 4 requests, 94.8% cache read, $0.00179, matching recompute. 4/4 pairs, 2 outputs with absent baselines, leftovers empty, no learned file. The scenario name says sonnet but the configured model is claude-haiku-5-5, as intended for this attempt."
  }
 ]
}