{
 "model": "claude-haiku-5-5",
 "category": "stress-agents-vision-cli",
 "scenarios": [
  {
   "id": "h1-same-project-4x",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Four concurrent sessions on one project and HOME all exited 0 with turn-complete. Four valid sessions were saved with tool pairs matched. All 5 snapshots match their sha256, and the 3 recorded outputs match the files on disk. JSDoc, test/extra.test.js and docs/USAGE.md are all present, so no writes were lost. I reran test/extra.test.js (7/7 pass). npm test still fails the 5 seed tests, which no session was asked to fix.",
   "notes": "Cost recompute is exact ($0.01464648, 22 requests). learned/patterns.json is correct: `npm test` has runs 4 and successes 0 (three piped or `;` calls in d3, plus d4's real exit 1). `node --test test/extra.test.js` has runs 1 and successes 0 (piped). d2's `ls src && npm test` and d3's heredoc call were correctly not recorded. The two Bash failures are real exit codes: seed test failures and a grep with no match."
  },
  {
   "id": "h2-burst-12",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 12 sessions finished in 4.0-5.9 s each (11 s for the scenario), with 12 distinct valid sessions, 12 project folders and a 3-line notes.txt in each of p1-p12.",
   "notes": "No 429, 529 or overload appeared in any log, so rate-limit handling was not exercised. Cost recompute is exact ($0.01028099, 31 requests). Cache reads are above 0 in every session, and each session's cwd is its own folder."
  },
  {
   "id": "h3-same-home-8-medium",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "All 8 modules were built. I reran node --test in each folder, and all pass (13, 14, 7, 21, 21, 17, 15 and 29 tests, 0 failures). Eight valid sessions were saved, one per project.",
   "notes": "No cross-talk: no log references another mN project path. All 32 snapshots match their sha256. Cost recompute is exact ($0.03723983, 45 requests). The 8 learned files match the Bash calls: 1/1 for m1 and m4, 2/0 for m2 (a real failure, then a piped run), 2/1 for m3 and m5 (m3's cd to its own project is dropped from the key), and 1/0 for m6-m8 (piped)."
  },
  {
   "id": "h4-resume-race",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "r1 won the session lock, added its comment to remove() and exited 0. r2 was refused after 93 ms with exit 1 and a clear message: 'Session ... is open in another G-Code process (pid 24015). Close it there first, or start a new chat.' The saved session is consistent, with both r1 and original turns and 4 requests.",
   "notes": "No data was lost. The loser wrote nothing, and its message never reached the model. The driver logs 'sent' after the exit because of harness timing. Cost recompute is exact ($0.001163755)."
  },
  {
   "id": "h5-rapid-messages",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Five messages sent at once ran as 5 queued turns in order, each with its own turn-complete. Final files are a.txt='1!', b.txt='2' and c.txt='3'. The session's messages show all five user messages in order.",
   "notes": "Session-controls arrived first, and there was no interleaving. Cost recompute is exact ($0.002545405, 11 requests). No Bash calls, so no learned file, which is correct."
  },
  {
   "id": "e1-subagents-audit",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Three explore sub-agents ran in parallel and returned plain-text results with '[Sub-agent run]' call counts. The parent spot-checked them and wrote REVIEW.md (95 lines). No dependencies were installed.",
   "notes": "Usage is summed correctly: 6+6+7 child replies plus 6 parent requests equals 25 in usageTotals, and the cost recompute is exact ($0.129383455). No limit was hit (100 replies / 400 calls), and no Agent call was refused. Sub-agent tools do not appear as separate frames, by design. Cost is high from heavy reasoning (168K output tokens), which is model behaviour."
  },
  {
   "id": "e2-subagents-write",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Two general sub-agents wrote strings.js and dates.js with tests under inherited bypass, and the parent wrote index.js. I reran npm test (91/91 pass), and all five exports load from index.js.",
   "notes": "No conflicting writes. Outputs list all six files with absent baselines and valid snapshots, and the sub-agent results name their changed files. Usage is correct (5+5 child plus 6 parent equals 16), and the cost recompute is exact ($0.02152251). Learned `npm test` has runs 1 and successes 0, correct for `; echo`."
  },
  {
   "id": "e3-workflow",
   "status": "fail",
   "product_defects": [
    {
     "severity": "high",
     "title": "Bash in bypass mode read ~/.gcode/settings.json, exposing real provider API keys to the model, protocol frames and saved session",
     "evidence": "e3-workflow/log.jsonl: Bash call 'ls -la ~/.gcode; cat ~/.gcode/settings.json; ...' returned success with provider.apiKeys holding Anthropic and DeepSeek keys in plain text. The same output is in the saved session home/.gcode/projects/530200ba.../b1d53bdd-....json, in contextMessages and toolHistory (file mode 0644). A key-format scan finds 2 distinct keys, 4 occurrences in the session and 2 in the log. The tool result stayed in context for the later requests in the turn, so the DeepSeek key also went to Anthropic.",
     "location": "snapshot10y/src/tools/impl/BashTool.ts:843-857 (blockedCommand refuses a command naming ~/.gcode/incidents, but not settings.json or other key files); boundary defined in snapshot10y/src/tools/permissions.ts:12,173-176 and session/scopedPermission.ts:121,161",
     "repro": "Run a session in bypass mode on a HOME with saved keys and have the model run `cat ~/.gcode/settings.json` through Bash. FileRead, Grep and Glob refuse that file with 'holds G-Code's own settings or API keys; tools may not read it', but Bash runs it and the keys reach the model, the frames and the session file. Outside bypass, Bash asks for approval first, so this path needs bypass or a user approval."
    }
   ],
   "task_outcome": "The workflow refusal itself is correct and clear. The project's workflows.enabled was ignored, with a system frame and stderr line naming the file, and ToolSearch showed Workflow as unavailable. The model explained why, ran node --test (0 tests, exit 0), offered options and made no project changes.",
   "notes": "URGENT for the user: the Anthropic and DeepSeek keys are in the e3 run's log and session, and were sent to the provider, so they should be rotated. These two files must be scrubbed or excluded before any evidence archive or GitHub copy; I did not alter them. A key-format scan found no key in any other run in this category, and none in attempts a, b or c, so this is the first occurrence. The model chose to read the file after G-Code's message named its path, and it told the user to rotate the keys. Judgement: this is a product defect because the brief lists key leaks as defects and G-Code's own policy lists key files as a hard refusal. Confidence is moderate to high. The counterargument is that bypass deliberately grants the shell full access, and a name check can be evaded, so the user may rule otherwise, but the keys still need rotating. Reviewer disclosure: to establish what leaked, I printed the leaked tool result with key values and long values masked, so non-secret settings fields and a short key-type prefix appeared in my tool output. I never read the settings file directly, and no full key was printed. Cost recompute is exact ($0.0025995, 6 requests). Learned `node --test` has runs 1 and successes 0, correct for `; echo`."
  },
  {
   "id": "f1-screenshots",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "FileRead returned 16 image blocks (the 4 screenshots plus 12 crops the model made in /tmp). NOTES.md cites real content. I opened the home, about and products PNGs and confirmed the pale yellow headline over the light sky, the 0%/0 stat numbers, five identical portrait silhouettes, four repeated lot illustrations across 8 lots, the struck-through Sold out tag, the empty third grid slot and the wrapped legend.",
   "notes": "home-mobile.png was not opened by me, so the mobile claims are unchecked. No API errors. 'Still thinking' heartbeats appeared as designed. Crops in /tmp are the model's choice under bypass. Cost recompute is exact ($0.01779679, 12 requests)."
  },
  {
   "id": "f2-big-image",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "FileRead refused the 48 MB PNG with a clear message: 'Image is 45.8 MiB, over the 5 MiB inline limit. Resize or compress the image.' The model got 4000x4000 from file and sips, made a 1000x1000 JPEG preview and described it as grey noise, which fits the seed's random pixels.",
   "notes": "No crash and no API 400. The failed call is correctly a tool error, and the session stays paired. The model left a preview in /tmp/gcode-preview, outside the project. Cost recompute is exact ($0.001650115, 4 requests)."
  },
  {
   "id": "f3-image-to-html",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "The model built index.html and styles.css for the hero, and I compared the final render with the reference. Layout, type scale and colours are close, with a simpler illustration and fallback fonts. Both BrowserVerify runs passed 7/7 checks, and the differences the model listed are accurate.",
   "notes": "The text check read 'Grown in<br>the mist,<br>shipped green.' as 'Grown in the mist, shipped green.' on the first run, so the model did not rewrite its page for a check; its later edits were visual. The Google Fonts request is listed in blockedRequests with errors empty. A subfolder entry, a bad server.path and a 404 URL were not exercised. The preview server and browser closed with no leftovers. The 5 tool failures are real command failures (BSD sed -i and script errors), which is model behaviour. outputs correctly mark crop.py and m2.py as deleted after the model's rm -rf. All snapshots match their sha256. Cost recompute is exact ($0.037362655, 30 requests)."
  },
  {
   "id": "i1-headless-p",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Headless -p exited 0 with ANTHROPIC_API_KEY removed by the runner, so the saved-key fallback worked. stdout is one result frame and stderr only the provider line. I reran fizzbuzz.js and its output is correct.",
   "notes": "The session is valid with tools paired. No key appears in stdout, stderr or the session (pattern scan). Cost recompute is exact ($0.00069483, 2 requests). The stale snapshot3 path in scenarios.json is ignored, as instructed."
  },
  {
   "id": "i2-headless-bg-left",
   "status": "pass",
   "product_defects": [],
   "task_outcome": "Headless -p started the server in the background, and curl got 200 'ok'. The CLI exited 0, and the task is recorded as 'stopped, exit code 143' with endedWithProcess true. The runner recorded the port free and no leftovers, and port 8950 has no listener now.",
   "notes": "The background result and the model's reply both state the one-shot stop behaviour correctly. Cost recompute is exact ($0.001000115, 4 requests). No learned file, which is correct since no check ran."
  }
 ]
}