{
  "model": "claude-haiku-5-5",
  "category": "build",
  "scenarios": [
    {
      "id": "a1-static-site",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Five-page bakery site completed; independent node:test 17/17, 64 local links across five HTML pages valid; raw curl checks returned 200, TaskStop stopped the server, and four BrowserVerify reports passed 31 assertions across 57 steps.",
      "notes": "Category-wide independent checks: all 10 raw logs parse; each has one turn-complete, driver exit 0, empty driver.err, no unmatched or duplicate saved tool pairs, sane complete usage with cache reads, matching final outputs/session data, valid snapshot hashes, empty recorded leftovers, and currently free original IPv4/IPv6 ports. Every saved request catalog has exactly 30 tools and all six bundled Skill descriptions, with no AskUserQuestion, DeferredToolCall, DSH tools/directory or GPCI runtime evidence. Original projects stayed byte-identical. Tests used freshly isolated HOME/USERPROFILE/TMPDIR and private Node v24.22.0; no installs, live model calls, settings access, GUI or product edits. Audit/check JSON and command exit logs are in each scenario/review16b-build-audit/. BrowserVerify reports were inspected, not independently rerun; this CLI gate gives no native desktop/PTY/DSH coverage. Costs below exactly match saved usageTotals using snapshot10u/src/shared/pricing.ts, are offline estimates rather than provider bills, and all ten have zero partial/unreported and zero long-prompt requests. This scenario has 43 paired calls, 48 tracked outputs with absent:true creation baselines, and four heartbeat frames that ceased before visible output continued. The model final prose says 52 browser checks; the actual reports contain 57 steps and 31 assertions, so the prose count is not used as evidence. Estimated USD 0.047915100, 18 requests."
    },
    {
      "id": "a2-vite-react",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Vite/React/TypeScript scaffold serves Hello from G-Code; independent TypeScript build and Vite production build both exit 0 with existing dependencies.",
      "notes": "11 saved/runtime pairs. Raw log lines 31-61 show Vite 8.3.5 on strict port 8705, successful HTTP/source checks and a running task at turn end; recorded CLI shutdown has no leftovers and the port is now free. Exact historical ~3-second shutdown timing was not independently recreated. App.tsx retains original and edited snapshots; absent:true is the session-creation baseline. React 19.3.1 and TypeScript 7.0.2 remained pinned. Initial reviewer copy incorrectly omitted dependency-internal dist directories, producing review-only failures; those logs are preserved and a fresh complete copy passed without changing originals or dependencies. See independent-checks-complete-copy.json. No BrowserVerify requested or run. Estimated USD 0.004135830, 9 requests."
    },
    {
      "id": "a3-vite-build-preview",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Three-origin coffee section built; independent TypeScript and Vite builds exit 0 and rebuilt dist hashes equal the original; raw index/JS HTTP checks returned 200 and TaskStop closed the preview server.",
      "notes": "12 saved/runtime pairs; 13 tracked outputs. The inherited checks text asks for scaffold baselines, but snapshot10u/src/session/outputs.ts:137-146,185,467-499 records the first change in a session: Bash scaffolding creates absent:true baselines. Both src/main.ts and src/style.css retain complete original scaffold version 0 and edited version 1, so no baseline loss occurred. Vite 8.3.5/TypeScript 7.0.2 unchanged. Initial reviewer copy omitted dependency dist directories; preserved failed logs were followed by successful checks on a fresh complete copy, not a product repair. No BrowserVerify requested or run. Estimated USD 0.004720350, 10 requests."
    },
    {
      "id": "a4-canvas-game",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Snake game with separate pure logic and injectable RNG completed; independent node:test passes 58/58. Two retained BrowserVerify reports each pass 18 assertions and record cleanup.",
      "notes": "27 saved tool pairs; 26 executed runtime uses plus the refused input result. Initial npm-test output showed failures while tail pipelines returned shell success; the model edited and retested to 58 passes, and no failure guard stopped progress. BrowserPlan empty expected.equals was correctly refused/Not run, never denied; all later valid calls completed. Five heartbeat frames cease before visible output. All 22 output snapshots verify. Estimated USD 0.047274085, 24 requests."
    },
    {
      "id": "a5-md-docs",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Zero-dependency Markdown builder completed; independent node:test passes 50/50, build exits 0, all six HTML files exactly reproduce original bytes, and 54 local links/anchors resolve.",
      "notes": "15 saved/runtime pairs; Changes includes all nine generated files (builder, package, tests and six pages), with matching hashes and absent:true baselines. Original content has duplicated headings and no fenced code because the harness printf seed consumed backticks; this is preserved seed behavior, not G-Code data loss. Fenced-code/list/emphasis/link behavior is exercised by passing project tests. Nine heartbeat frames cease before visible output. Actual driver/session effort is high despite stale medium in scenarios.json. Estimated USD 0.037921925, 9 requests."
    },
    {
      "id": "a6-express-api",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Express notes CRUD and static frontend completed; independent loopback probe passes 10 status/data checks, exits 0, closes its owned server, and its ephemeral port is confirmed closed.",
      "notes": "11 saved/runtime pairs; raw log records npm installation, create/list/read/update/delete/404/validation/static curl checks, TaskStop and connection refusal afterward. Existing Express 5.3.0 retained. package.json still has npm-init placeholder test, so it was not treated as an implemented test suite; the reviewed independent probe supplies the check. Initial reviewer copy omitted dependency dist directories; preserved failure logs were followed by a successful fresh complete-copy probe, without install or original changes. Four tracked outputs verify. Estimated USD 0.006169325, 10 requests."
    },
    {
      "id": "a7-chinese-site",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Four-page Chinese restaurant site completed; independent check.mjs exits 0, all four lang/title checks pass, 31 local links resolve, and the original BrowserVerify passes seven assertions.",
      "notes": "17 saved/runtime pairs and 16 valid tracked outputs. Frames, saved session, project text and Chinese final summary preserve UTF-8 with no U+FFFD. The raw negative checker fixture deliberately uses lang=en and a missing title; its real exit 1 correctly remains failed, not refused/denied, while the actual four-page check passes. The negative shell used set -e, so its later cleanup command was skipped; no product process/listener leak is recorded. Estimated USD 0.012556660, 12 requests."
    },
    {
      "id": "j1-todo-fullstack",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "JSON-backed todo REST app and add/toggle/delete/filter frontend completed; independent store/HTTP tests pass 54/54 with ephemeral loopback servers; raw endpoint checks and TaskStop succeed, and BrowserVerify passes 26 assertions.",
      "notes": "24 saved tool-use/result pairs; 23 runtime executed uses/results. One malformed-JSON BrowserPlan never executed and is persisted refused with its synthetic paired result, matching snapshot10u/src/engine/queryEngine.ts:1809-1815; the separate call that executed with nonexistent index.html remains failed. Correcting entry to public/index.html then passes; this is model argument recovery, not a product defect. Skill browser-verify loads without approval/denial. Six waiting/thinking heartbeat frames stop before visible output; 18 tracked outputs verify. Independent test processes/groups are gone. Estimated USD 0.051196005, 18 requests."
    },
    {
      "id": "j2-cli-tool",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "CSV statistics CLI completed; independent node:test passes 33/33, CLI and generator exit 0, regenerated CSV is byte-identical, and Python csv independently confirms 2,000 records and eight columns.",
      "notes": "Eight saved/runtime pairs, five tracked outputs with valid snapshots. Quoted commas, escaped quotes and embedded newlines are exercised; 2,297 physical lines correctly represent 2,000 data rows. Original raw Python/Node statistics agree, and independent sample IDs span 1-2000. No background server or BrowserVerify was requested or run. Estimated USD 0.012351170, 9 requests."
    },
    {
      "id": "j7-sonnet-sanity",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "TABLE.md and verifier completed; independent private-Node verification exits 0 and confirms exactly eight planetary data rows.",
      "notes": "Four saved/runtime pairs and two complete tracked outputs; the actual configured/saved model is claude-haiku-5-5 at high effort despite the legacy sonnet-sanity ID and stale medium scenario default. This validates table structure, not current astronomical moon-count facts. No background task or BrowserVerify was requested or run. Estimated USD 0.002029425, 4 requests, the smallest calibration run in this category."
    }
  ]
}
