{
  "model": "claude-haiku-5-5",
  "category": "stress+agents-vision-cli",
  "scenarios": [
    {
      "id": "h1-same-project-4x",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Four concurrent drivers completed into four distinct valid sessions; JSDoc, extra tests and docs/USAGE.md survived. All 22 saved tool pairs and four output/baseline snapshots verified. Independent Node v24.22.0 test run: 5 pass, 8 fail, exit 1; untouched seed independently gives 0 pass, 5 fail, exit 1.",
      "notes": "The inventory implementation matches the seed after removing comments; original tests and report.js are unchanged. Five original failures plus three new lowStock boundary failures expose seeded task-code bugs, not lost writes or G-Code failures. d4 correctly reports its executed Bash failure. All four driver exits are 0. Common checks across all 13 assigned scenarios: valid session JSON and paired saved calls; appropriate approvals/terminal states; 30 advertised tools including the six-skill catalog; no AskUserQuestion, DeferredToolCall or DSH tool/directory; no unparsed protocol rows; empty recorded process/listener leftovers and empty .err files. Independently hashed 102 scenario output/baseline .snap files, not frozen source files. Recorded costs match saved usageTotals under snapshot10u pricing, including cache read/write and long-prompt splits; all these sessions report complete usage. Costs are offline estimates, not provider invoices. Own execution used isolated HOME/USERPROFILE and reviewed copies/existing runtimes; no new model request, dependency installation, real settings, installed app or native PTY/DSH coverage. Saved-usage estimate independently recomputes to USD 0.012008210 for 17 requests, matching result.json."
    },
    {
      "id": "h2-burst-12",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "All 12 parallel projects completed, each with its own valid session, notes.txt, FileWrite/FileRead pair and matching output snapshot; 24 pairs and 12 snapshots verified. Recorded scenario duration is 11 seconds.",
      "notes": "All 12 exits are 0; no 429, 529, overload, reply-restart or cross-project output was observed. Each notes file contains three numbered items and was read back. Coffee factual/copy quality is model output, not independently fact-checked here. Actual configs and controls use High effort despite historical low-effort scenario text. Saved-usage estimate independently recomputes to USD 0.009195600 for 29 requests, matching result.json."
    },
    {
      "id": "h3-same-home-8-medium",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Eight valid sessions map to eight correct cwd hashes and original task messages, with no cross-project paths/messages found in saved context. All 51 pairs and 27 output snapshots verified. Independent Node24 tests pass for all eight modules: 11+13+10+17+19+14+13+25 = 122 tests, eight exits 0.",
      "notes": "Slugify, Roman numerals, Luhn, word count, date difference, temperature conversion, queue and LRU artifacts remain separated. All live exits are 0; recorded duration 34 seconds. Actual controls/configs use High, despite the historical medium scenario name. Passing generated tests is bounded artifact validation, not an exhaustive proof of each module. Saved-usage estimate independently recomputes to USD 0.030146310 for 37 requests, matching result.json."
    },
    {
      "id": "h4-resume-race",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Initial session and r1 resume completed; r1 added the remove() comment. r2 was refused before tool execution with exit 1 and the explicit open-in-another-G-Code-process message. One valid session preserves initial and winning turns, two paired calls and both output/baseline snapshots.",
      "notes": "r2.jsonl line 2 names session 9334ec47-3dd3-4b5c-8e71-8e65c6ed2150 and asks to close the other owner or start a new chat; its summary records exitedEarly and exitCodes [1]. This is the intended lock refusal, not a successful second turn or data loss. Snapshot10u/src/state/store.ts:254-260 confirms ownership semantics. Independent artifact test exits 1 with the same five failures as the untouched inventory seed; comment-stripped implementation and existing tests match the seed. Saved-usage estimate independently recomputes to USD 0.001256560 for 4 requests, matching result.json."
    },
    {
      "id": "h5-rapid-messages",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "All five messages sent at one timestamp completed in order with five turn-complete frames. Independent byte checks give a.txt=\"1!\", b.txt=\"2\", c.txt=\"3\"; saved user requests remain in order. Nine paired calls and four output versions verified.",
      "notes": "Controls report bypass/High before any tool execution. Session-memory additions to later wire requests are labeled runtime context and preserve the queued requests. Live exit 0, no interleaved writes, lost message, error or leftover. Saved-usage estimate independently recomputes to USD 0.002479075 for 11 requests, matching result.json."
    },
    {
      "id": "e1-subagents-audit",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Three parallel explore subagents returned accessibility, performance and copy reviews; parent produced REVIEW.md without changing seeded source. Child summaries show 22/35/26 tool calls in 6/10/8 replies; saved 39 requests reconcile to 15 parent + 24 child requests. Four REVIEW snapshots verified.",
      "notes": "Three parent FileEdit calls with malformed JSON are saved as refused/not executed and paired in context; later valid edits complete. The 18 executed frame pairs and 21 saved pairs are therefore consistent. Explore calls required no approvals. Source retains the 100-reply and 400-call limits; this run did not reach either limit or exercise partial-work-on-limit. Build not run: both seed and result intentionally lack node_modules, and the task forbids installation; scripts require tsc and Vite. Findings remain a static model audit, not measured accessibility/performance or a successful build. Saved-usage estimate independently recomputes to USD 0.207431660 for 39 requests, matching result.json."
    },
    {
      "id": "e2-subagents-write",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Two parallel general subagents wrote disjoint strings and dates implementations/tests; parent supplied package.json and index.js. Parent Changes ledger contains all six files with absent baselines and seven hash-verified versions. Independent Node24 run passes all 39 tests, exit 0.",
      "notes": "Bypass permission inherited without approval prompts. Each child made five tool calls in six replies; total usage reconciles to six parent + twelve child = eighteen requests. Child result text lists exactly its two changed files; no conflict or lost write observed. Both limits remain unexercised. The model documents its truncate whitespace/length choice and relative-time thresholds; these are artifact semantics, not engine defects. Saved-usage estimate independently recomputes to USD 0.020707035 for 18 requests, matching result.json."
    },
    {
      "id": "e3-workflow",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Workflow remained unavailable despite the project enable flag. The system warning and final response clearly explain that only user settings can enable it. ToolSearch and source inspection ran; no Workflow or workflow agents ran and no source files changed. Independent Node24 --test exits 0 with zero tests.",
      "notes": "The zero-test result is explicitly not functional test coverage. User-only enablement matches the current contract; stale scenario wording does not authorize project-level activation. No user settings or run-home settings link was read during review. Nine saved tool pairs, zero outputs, no approvals or leftovers. Saved-usage estimate independently recomputes to USD 0.001978515 for 4 requests, matching result.json."
    },
    {
      "id": "f1-screenshots",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Opened all four original PNGs and compared NOTES.md with visible content. Original reads/crops returned image blocks; 22 image reads were handled with the documented 20-image retained limit and an explicit two-image removal notice. Retained original image payload hashes match their PNGs; NOTES.md snapshot verifies.",
      "notes": "Concrete visible findings include the pale yellow hero text, faint eyebrow, repeated lot artwork, About portrait silhouettes, and Products card alignment. Model artifact limitations: NOTES.md wrongly assigns About people/stat sections to Home; it says three illustration types while listing four classes; it admits mobile hero/header/footer coverage is incomplete after incorrect sips cropping. These are model observation/coverage errors, not transport failures. No crash or provider 400. Authorized PNG encoding-prefix coincidences were not treated as credentials, and original images/JSON were unchanged. Saved-usage estimate independently recomputes to USD 0.026779405 for 13 requests, matching result.json."
    },
    {
      "id": "f2-big-image",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Independent PNG header/stat check confirms huge.png is 48,018,658 bytes and 4000x4000. Executed FileRead returns the clear 45.8 MiB over 5 MiB inline-limit error. The model creates a 1024x1024 preview; its successful image block is 2,504,308 bytes. Session completes without crash or API 400.",
      "notes": "Saved tool history correctly records the executed read error as failed, followed by a successful preview read; this is distinct from a pre-execution permission/invalid-input refusal. Snapshot10u/src/tools/impl/FileReadTool.ts:216 implements the size guard. Five saved pairs, live exit 0 and empty leftovers. Review did not use or alter the original /tmp preview; dimensions and retained payload were checked from immutable evidence. Saved-usage estimate independently recomputes to USD 0.001602305 for 4 requests, matching result.json."
    },
    {
      "id": "f3-image-to-html",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "index.html/styles.css recreate the hero with acknowledged illustration/font differences. All nine original BrowserVerify runs passed, including text on the first attempt; seven exercised a subfolder entry. Independent Node24 + bundled Playwright 1.64.1/Chromium check at 1440 and 390 widths passes HTTP 200, loaded CSS, one visible h1, normalized rendered title and no document overflow; exit 0, owned browser stopped and port free.",
      "notes": "The h1 retains its br elements; observed text is exactly \"Grown in the mist, shipped green.\" Page changes refine visuals, not compensate for a text-check failure. Invalid BrowserPlan selector placement is refused; two actually executed Bash errors (measurement IndexError and sed misuse) remain failed, then recover. No bad server.path or 404 navigation was exercised. Google Fonts requests are blockedRequests with errors=[], not page errors; independent review also blocks external network. The final deliverable uses fallback fonts offline, and illustration fidelity/placeholder navigation remain model-artifact limitations. All 75 saved pairs and 40 snapshots verify. Block-edge/display:none/multiple-match text cases were not separately exercised by this live artifact. No native UI coverage claimed. Saved-usage estimate independently recomputes to USD 0.159301330 for 56 requests, matching result.json."
    },
    {
      "id": "i1-headless-p",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Direct headless -p evidence records exit 0, valid result JSON and a saved session with two paired FileWrite/Bash calls. Independent execution of reviewed fizzbuzz.js on Node24 exits 0 and matches all 30 expected FizzBuzz lines exactly; one output snapshot verified.",
      "notes": "Headless stdout/stderr contain no observed credential material; stderr is only the provider banner. No denial occurred, so no permission_denials payload is required. Authentication settings/key fallback was not re-read under H3, and no new authenticated invocation was made. Recorded afterExit port is free and leftovers are empty. Saved-usage estimate independently recomputes to USD 0.000733250 for 2 requests, matching result.json."
    },
    {
      "id": "i2-headless-bg-left",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Saved calls actually start LISTEN_ON=8950 node server.mjs in background task bg-f6362532/process group 13381, receive HTTP 200 with body ok via curl, and inspect TaskOutput. Headless exits 0; original afterExit records port free and no processes/listeners. Independent current lsof/ps checks find neither port-8950 listener nor PID 13381.",
      "notes": "The tool result and final answer explicitly say one-shot completion stops the background server. Snapshot10u CLI cleanup awaits backgroundShells.stopAll. No background service was restarted during review. Current lsof and ps both exit 1 with empty output (no match), saved in review16b/cleanup-check.json. Exact subsecond SIGTERM latency is not recorded, so the approximately 1.5-second timing is not independently claimed. Four saved pairs, no permission denial or leak observed. Saved-usage estimate independently recomputes to USD 0.001182725 for 5 requests, matching result.json."
    }
  ]
}
