{
  "model": "claude-haiku-5-5",
  "category": "background",
  "scenarios": [
    {
      "id": "d1-two-servers",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Both servers started as tasks bg-fd377919 (8810) and bg-253ffdcb (8811); curl got 200 from both; TaskStop stopped only the first (exit 143); 8810 then refused connections and 8811 still returned 200; TaskOutput reported stopped, exit code 143 and running. Checked in frames 13-43, the saved session and the task records.",
      "notes": "Exit code 0, turn-complete, driver.err empty. Saved pairs 8/8. Recomputed cost $0.002082455 equals reported; cache read ratio 0.97. The second server's record reads stopped, endedWithProcess true. result.json leftovers empty, and I found no listener on 8810/8811 and no http.server process. 31 tools with the 6-skill catalog, no mcp or DeferredToolCall. Model oddity only: its final reply says the second port was 8811 because the command specified it (the driver had already filled in the number)."
    },
    {
      "id": "d2-bad-bg-command",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "definitely-not-a-command reported as failed, exit 127, with the 'command not found' line; the sh -c command reported as failed, exit 3, with starting and crashed in its output; tasks frames show failed with exit codes 127 and 3; TaskOutput reads 'failed, exit code 3'. No hang.",
      "notes": "Exit code 0, pairs 3/3, toolHistory failed, failed, complete. toolFailures 2 are the two calls that ran and exited non-zero, which correctly stay failed. Recomputed cost $0.001083895 equals reported. Exit code stated three times in the model-facing result is the known, deliberate wording and reads fine. stdout and stderr are merged in the task output, and the model said so accurately. Leftovers empty."
    },
    {
      "id": "d3-unicode-bg",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "All six lines of Chinese, emoji and the ZWJ sequence are intact in the Bash start result, every tasks tail, both TaskOutput results and the saved session; no U+FFFD in log.jsonl or the session; outputBytes 324 is exactly 6 x 54 bytes; final state completed, exit code 0.",
      "notes": "Raw ANSI escapes are kept unchanged in frames and on the wire (12 ESC bytes in the TaskOutput results), which is sane for a request for the exact text. The live run wrote whole lines, so it did not split a character across reads; I probed that offline under a scratch HOME with backgroundShells.start on a command splitting 3-byte and 4-byte characters across writes, and the output came back intact with no U+FFFD. Neither TaskOutput read happened while running: the model sent both after the 2 s start look and they ran at 925722, 46 ms after the task ended at 925676. Output while running is shown by the start result (lines 1-4) and the running tasks tails. Model misstatements, G-Code correct: its reply says the first read came back while running and showed 1 through 4, which was the Bash start result, and it says ESC bytes were dropped, which they were not. No 'while you worked' notice, correctly, since TaskOutput showed the end. Recomputed cost $0.001194995 equals reported. Leftovers empty."
    },
    {
      "id": "d4-watcher-left-running",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "node --watch started in background, curl returned ok, server.mjs edited to updated, the watcher logged the restart and curl returned updated; the task was still running at turn end and was stopped when the session closed. Port 8825 free and no node --watch process remains.",
      "notes": "The model put `&` inside a run_in_background command; the task stayed running while the group lived, as designed (backgroundShells.ts finish). The record reads 'stopped, exit code 0' with endedWithProcess true. The runner's ps check cannot see this command line because it holds no run path, so the check that counts is the empty listener list 5 s after exit plus my own later ps and lsof, which found nothing; the later check alone does not prove timing. watcher.log is tracked with 2 versions, ending at 159 bytes. Pairs 5/5. Recomputed cost $0.001495635 equals reported."
    },
    {
      "id": "d5-max-bg",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Eight sleeps started; call 9 returned 'Error: 8 background tasks are already running. Stop one with TaskStop first.'; call 10 was refused by the repeat guard; eight TaskStop calls each reported exit code 143; TaskList and the final tasks frame show all eight stopped.",
      "notes": "toolHistory confirms the round's rule: call 9 failed (ran and returned an error), call 10 refused. The result frame with no tool-use frame for call 10 is by design, and saved pairs are 19/19. Summary toolFailures of 2 counts success:false, including the refused call. No 'while you worked' notices after the model's own TaskStop. Limit of the evidence: the runner's ps check cannot match `sleep 300`, and my later check came long after 300 s, so that proves nothing alone; the stop evidence is exit 143 for each group plus the group-kill code. Recomputed cost $0.00249102 equals reported."
    },
    {
      "id": "d6-interrupt-resume",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "SIGINT was sent on the 6th call's tool-use frame: the FileEdit was not run (index.js still lacked /time on resume), and the error frame read 'Stopped before this turn finished.'. The resumed session continued with no 400 and no repeat of npm init, install or the file write, added /time, verified both routes with curl, stopped the server and confirmed port 8835 free. index.js holds both routes.",
      "notes": "Saved turn: synthetic is_error result 'Not run: the turn was stopped before this FileEdit call started, so nothing was executed.', then '[The user stopped this turn before it finished...]'; state cancelled; pairs 13/13; usage spans both processes (11 requests); recomputed cost $0.00382345 equals reported. Both CLI exits are 0, as the non-headless path intends (cli.ts: 128+N is headless only). The model's `node index.js &` in a foreground Bash was adopted as bg-62a5c1df and told to the model correctly; TaskStop gave no exit code for it because an adopted group's status is unknown. What this run did NOT exercise: TaskStop finished before the stop, so no TaskStop was itself interrupted and no shell was still running at the interrupt; kill-on-interrupt and the resume notice about ended shells were not shown here (the record is reported true, correctly no notice). No tasks frame showing bg-62a5c1df as stopped was sent before that process exited, because the 250 ms debounce was cleared on close (repl.ts:602); the desktop marks the list stopped on process end (useChatSession.ts:278, 483), read in code only, so this is not a defect. No 'The operation was aborted.' text anywhere. Leftovers empty."
    },
    {
      "id": "d7-sigkill-orphan",
      "status": "pass",
      "product_defects": [],
      "task_outcome": "Server started and verified with curl, foreground sleep 200 running when the CLI got SIGKILL: result.json shows killed true and serverGoneMs 1085, under the ~5 s limit; driver exit code 137 with exitedEarly true; leftovers empty.",
      "notes": "Saved checkpoint: synthetic 'No result was recorded: the process ended before this call finished...' plus the incomplete-turn note, pairs 5/5. toolHistory holds the sleep as running, and store.ts restoreSession turns that into cancelled on load (code read only, no resume in this run). The task record stays running, which the design reads as the process having died. The leftover .lock names the dead pid; takeover was read in acquireSessionLock, not exercised. Summary errors is empty rather than holding an error, because the CLI died with no frame; the driver records outcome exited. The runner's checks cannot see the foreground sleep (no run path in its command line, and no port), so I probed it offline under a scratch HOME with the snapshot's runSubprocess (group, killOnExit, as BashTool calls it) and backgroundShells.start: after kill -9 of the parent, both groups' sleeps were gone within 1 s and both watchdogs exited by 3 s. Probe removed, nothing left behind. Effects of the round 18 change: no DiagnosticReports, log show, unexplained-kill, network incident or sandbox-exec text in log.jsonl or saved state, and no incidents folder, in any d scenario; a command killed by SIGKILL reported exit 137 and failed as a background task and exitCode null with signal SIGKILL in the foreground subprocess result in the same probe. A `log show` process cannot be ruled out after the fact from logs alone, though guard is undefined without Guillotine Mode. Recomputed cost $0.00126856 equals reported."
    }
  ]
}
