{
 "model": "claude-haiku-5-5",
 "category": "build",
 "scenarios": [
  {
   "id": "a1-static-site",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All five pages, shared css/js, lightbox, js/validate.js and fetch-rendered menu were built. My rerun of node --test passed 22/22, and no local links are broken. The agent's curl checks returned 200, TaskStop ended the server (exit 143), and port 8700 is free.",
   "notes": "43 tool calls in one turn with 43/43 pairs. Turn-complete, exit 0, driver.err empty, leftovers empty. All 41 outputs have baseline absent:true, and latest versions match disk. Only .gcode/verification/.gitignore is absent from outputs, as in the other BrowserVerify runs. Three BrowserVerify reports passed and match their results. The model called Skill browser-verify, which needed no approval; no trigger block, since the user text has no trigger. Eight heartbeats about 15 s apart, none after output began. Learned `node --test` shows runs 4, successes 0, correct because all four runs were piped. Cost recompute matches $0.05954719. The model wrote a stray 生 in about.html and fixed it itself."
  },
  {
   "id": "a2-vite-react",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Vite React TS app scaffolded and installed, and the heading changed to 'Hello from G-Code'. Curl of /src/App.tsx served the new text. The dev server was left running as asked. My `npm run build` in a scratch copy exited 0.",
   "notes": "10/10 pairs. The last tasks frame during the turn shows the server running, and there are no leftover processes or listeners after exit. The runner samples leftovers 5 s after the driver exits, so the 3 s limit in the check was not measured separately; port 8705 was free when I checked. Learned patterns correctly absent (no checks run). The model wrote /tmp/index.html outside the project under bypass, which is model choice. Cost matches $0.00317557."
  },
  {
   "id": "a3-vite-build-preview",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Vanilla-ts app scaffolded with a coffee origins section. My `npm run build` in a scratch copy exited 0. The agent's curl got 200 for index.html and the JS asset, and TaskStop ended the server (exit 143) with the port confirmed closed.",
   "notes": "11/11 pairs. The stale check says the baseline holds the scaffold, but the scaffold was created in this session, so baseline absent:true is right. versions[0] of src/main.ts (2464 bytes) and src/style.css (5060 bytes) hold the original scaffold; versions[1] holds the edits, including the style.css edit made through Bash `cat >>`. dist/ is missing from outputs because the project's .gitignore lists it. No learned file: the only `npm run build` follows a here-document, so it is unrecorded by design. Cost matches $0.0044277."
  },
  {
   "id": "a4-canvas-game",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Snake built as index.html, game.js and logic.js with an injectable RNG. My rerun of npm test passed 63/63. No local links are broken, and the second BrowserVerify passed 33 checks.",
   "notes": "The failure guard did not interfere with the test-edit-retest loop (10 failing tests, then fixes, then 63/63). The first BrowserVerify found a real overflow at phone width and settled failed with a single 'Error:' prefix. The model fixed the CSS and it passed. 27/27 pairs. Learned `npm test` shows runs 4, successes 0, correct: the piped runs at 159, 168, 177 and 299 s count, and the `node --check ...` call is not recorded. Heartbeats stop when output starts. The model looked for a local Chrome under /Applications, harmless. Cost matches $0.06019376."
  },
  {
   "id": "a5-md-docs",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Zero-dependency build.mjs produces six site/*.html pages with layout, nav and TOC. My rerun of npm test passed 45/45, `npm run build` exited 0, and no local links are broken. BrowserVerify passed 12 checks.",
   "notes": "The first BrowserPlan ('Duplicate step name') settled refused with 'Not run: invalid input', and its tool-result frame has no tool-use frame, by design; 25/25 pairs in the saved session. Outputs hold 16 entries, all baseline absent:true, with the unchanged seed content/*.md correctly excluded. Harness note: the seed's backticks were eaten by the shell, so content/*.md is odd, not a G-Code issue. Heartbeat moved from 'Still waiting for the model (20s)' to 'Still thinking', then stopped at output. Learned `npm test` runs 2 and `node --test test/build.test.mjs` runs 1, successes 0, correct; the `npm pkg set ... && npm test` call is unrecorded by design. The driver and result record effort high, not the medium in scenarios.json, consistent with the brief. Cost matches $0.038122395."
  },
  {
   "id": "a6-express-api",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Express notes CRUD and static page built. The agent exercised every endpoint and the 404 case, and TaskStop ended the server (exit 143). My own smoke on port 18725 got index 200, POST, list, PUT, DELETE 204 and 404 after delete; I stopped it and confirmed no listener.",
   "notes": "9/9 pairs, 2 tasks frames, leftovers empty, no learned file (no checks run). No BrowserVerify used, which is fine. Cost matches $0.005293795."
  },
  {
   "id": "a7-chinese-site",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Four-page zh-CN site with a shared CSS file and menu.json prices in CNY. My rerun of node check.mjs exited 0, all pages have lang zh-CN and a title, no local links are broken, and menu.json parses. The final summary is in Chinese.",
   "notes": "Zero U+FFFD in frames, session or project, and no mojibake in the final summary, the HTML lang and title values, or the Chinese frame text I sampled. Three failures are correctly 'failed': a deliberate negative test under `set -e` (Bash exit 1), and two BrowserVerify runs caused by the model's selectors. One hit strict mode with 3 elements; the other used :last-child although a note <p> is the last child, which I confirmed in menu.html. Plan 3 passed. The model removed its temp dir with a $TMPDIR glob guarded by file checks, its own choice. It also typed a stray 'quot;' in menu.json and fixed it. 22/22 pairs, no skill block, no learned file (`node check.mjs` is not a check). Cost matches $0.02045994."
  },
  {
   "id": "j1-todo-fullstack",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Node http server, JSON store, vanilla frontend and tests built. My rerun of npm test passed 43/43. The agent's curl checks worked, and TaskStop ended the server with port 8955 confirmed free. My smoke on port 18955 passed POST, PATCH, filter and DELETE; I stopped it and confirmed no listener. BrowserVerify passed 36 checks.",
   "notes": "The first BrowserPlan (unknown key selector) settled refused with 'Not run', with no tool-use frame by design. The second, entry index.html, returned 'Error: Entry must be a project HTML file...' and settled failed. It comes from inside BrowserPlanTool.call() (browser/validation.ts:16), reads 'Error:' and saved nothing, so I judge it 'ran and failed', fairly sure. It is close to an input refusal, though: the same check in BrowserVerify's checkPermissions would settle refused, so the user may want to align them. The message is accurate and the model fixed the path at once. TaskStop reported exit 0 because npm start exited cleanly on the signal; status is stopped and leftovers are empty. 26/26 pairs. Learned `npm test` runs 1, successes 0, correct. Cost matches $0.0410276."
  },
  {
   "id": "j2-cli-tool",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "csvstat.mjs handles quoted fields and commas in quotes. sample.csv has 2,000 data rows (Python csv confirms; 2,212 physical lines because of embedded newlines). My rerun of npm test passed 36/36, and my run of the tool on sample.csv exited 0 with stats matching the agent's Python cross-check.",
   "notes": "10/10 pairs, no tool failures, leftovers empty. Learned `npm test` runs 2, successes 0, correct because both runs were piped; `npm run generate && ...` is not recorded. Three heartbeats, none after output began. Cost matches $0.019487285."
  },
  {
   "id": "j7-sonnet-sanity",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "TABLE.md has the 8 planets with diameter and moons. My rerun of node verify-table.js reported 8 data rows and PASS.",
   "notes": "Small baseline: 3 requests, 4 tools, 4/4 pairs, cacheReadTokens 37,777 (ratio 0.93), $0.00163357, matching the recompute. 31 tools including Skill and TeamMemoryWrite, as in every run in this category; no DeferredToolCall, AskUserQuestion or mcp tools. No ~/.gcode/auth or dsh folder and no GPCI text in frames or sessions in all 10 runs; no key-shaped strings in the logs, sessions or project files I scanned (node_modules skipped)."
  }
 ]
}