# G-Code for Mac: the test evidence, as text Rounds 10b to 21 of G-Code for Mac's 58-scenario gate, run on one Mac from 29 October to 8 November 2026. https://o-research.tech/results-mac summarises them; these are the files behind that page, as text, with the changes listed below. ## What's here - `gcode-mac-round10b.tar.xz`, `gcode-mac-round11.tar.xz`, `gcode-mac-round12.tar.xz`, and `gcode-mac-round13.tar.xz` to `gcode-mac-round21.tar.xz`: one archive per round, twelve in all. The first three are byte for byte the ones published on 1 November. - At the top, the round's records. `README.md` (in Chinese, the owner's working language); `evidence/` (final results, costs, coverage gaps, model notes, the test suite); `process/` (every run, every scenario's result with the reviewer's notes, defects, changes, decisions, test-script problems, test runs); `raw/` (the reviewers' verdicts as written, among them the fix lists and round summaries of round 10b's first runs; the 58 scenario definitions; file hashes of each frozen snapshot; test-suite summaries; the Guillotine Mode log). - `runs//`: one run of all 58 scenarios, with `status.json` and `runner.out`. A scenario folder usually holds `result.json` (time, cost, cache reads, leftover processes and ports), `driver.config.json` (the task and its settings), `driver.summary.json` (sessions, turns, tool calls, failures, approvals, handoffs, exit code), `log.jsonl` (every protocol frame except streamed text: each tool call and its output, approvals, status) and any scripts the reviewer wrote to check it. Scenarios with several drivers (h1–h3) keep a `.jsonl` and `.summary.json` per driver instead; the headless ones (i1, i2) keep `headless.json`; `verification/` holds the browser checks' reports where one ran. - Rounds 13 to 21 keep this layout. Every gate attempt, counted or not, is its own folder under `runs/`; rounds 15 and 20 also have runs with DSH on (`r15-dsh`, `r20-dsh`) and round 14 one extra scenario (`haiku-r14c-extra`). Round 13 also has `baseline/` and `gate/` (pattern counts of gate logs, this round's against earlier ones) and `local/` (its local-model runs). Round 16's `raw/` is the largest: its release attempts, the check of every claim in the documents, and the wiring audit. - Round 11 also has `runs/haiku-r11-invalid/`: its first attempt, void and not counted ($0.0037). The test script still imported a snapshot that had been moved, so most scenarios stopped within seconds (d7 ran for 7 minutes). It wasn't in the evidence package; it was copied from the test harness by the harness's own rules. Its empty driver summaries are listed in `MANIFEST.json`. - Round 10b's `runs/haiku-1/` also has `runner-rerun.out`. After a driver fix, Haiku run 1's g7, h1, h4, j5 and j6 were run again; the counted run ($1.4390, `status.json`) has the reruns, and `runner.out` is the first pass ($1.4120). The first attempts of those five weren't kept. `evidence/costs.json` lists the first pass's $1.4120 as a dry run, but 53 of its scenarios are the counted run; what went uncounted is those five first attempts, about $0.048 (from the two runner logs and `status.json`). - `MANIFEST.json`: each archive's size and SHA-256, and counts of everything changed or left out. - `WITHHELD.txt`: what was left out, and why. Unpack with `tar -xJf `. Check a download against `MANIFEST.json` with `shasum -a 256 `, and the files inside with `shasum -a 256 -c SHA256SUMS` from the unpacked folder. ## Left out, to keep it light - **The saved sessions**: 5,761 files, 888 MB. They repeat the tool calls in the protocol logs, carry G-Code's system prompt and are mostly screenshots. They also held the models' written replies, which the protocol logs don't; the reviewers' notes describe each outcome. - **Every image**: 439 files, 116 MB. The browser checks' screenshots and screenshots of the installed app. - **Files that aren't text**: a binary, an archive or a font would be left out and counted the same way. These twelve packages had none. Counted by round in `WITHHELD.txt` and `MANIFEST.json`, not listed one by one. Two single-scenario terra trials in round 10b (j7 and f2, not counted) kept only their cost; there are no files for them. ## Withheld Scenarios c1–c4 and j4 have the model read a seed folder that is an older copy of G-Code's own engine source, GPCI's code included. Their protocol logs are withheld so that source isn't published, and so is `reviewer-rebuilt-parts.txt` in c3 of terra 2–5: a reviewer's rebuild of the handoff summariser's requests, which quotes the session. Their `result.json`, run summaries, configs and the reviewers' scripts are included, and the reviewers' verdicts on them are in the records. `WITHHELD.txt` lists each file by name. Two more kinds, also listed by name, from round 16: - **Records of where copies are kept**: the harness backup manifest, the record of the kept archives, the backup's dry run, and the backup script in its three copies. Six files. They exist to name those places. - **Listings of the machine's processes**: eleven files in which the release checks recorded every process running before each install and screenshot. They name other software on the test machine. The checks' results are in the files around them. ## What was changed, and nothing else 1. **GPCI.** GPCI has been temporarily removed from G-Code for Mac, and its details aren't published. In round 11's records, what the removal took out and changed, the folders it touched, its command-line flags, a codename, and reviewers' remarks on its parts; in round 10b, a flag in the scenario definitions, a field name in one reviewer's script, and the lines for GPCI's own files (name and hash) in the snapshot hash lists. Each is replaced by `[removed for publication: …]`. Rounds 13 to 21 say only that GPCI existed, was removed and isn't found; a reviewers' brief that listed its parts, the note on how test fixtures were edited for its removal, and a field name in one ledger are replaced the same way. 2. **The test machine and its owner's files.** The home folder became `~`. The test harness's working folder became `/tmp/run`, and the temporary folder above it `/tmp/harness` (in haiku-7's f1 the model wrote its crops there, outside the harness's folder); where two reviewers quoted the harness folder's id, it is replaced too. macOS's per-user temporary folder became `$TMPDIR`, and the machine's host name `test-mac`. Where a full copy of G-Code is kept, the Trash folders old builds were moved to and the display's size are removed. In haiku-5's j3 a model listed the machine's shared `/tmp`, and in haiku-1's g2 one printed the machine's `PATH`; both outputs name other software on the machine and are replaced by a marker. The commands themselves are kept. From round 13 on, the repository's folder became `$REPO`, the test folder `$GATE` and the local models' folder `$MODELS`; any other `/Users/` (a model's invented path, a test's example) became `/Users/`. The owner's other folders and the project the installed app had open are replaced by a marker. The machine's model, chip, memory and OS build are removed (`[removed for publication: the test machine's model, chip, memory and OS build]`, or "the test machine"), as is its network address. Where old builds went, and where backups and full copies are kept, are removed. A service of the owner's that held a port during two gates became "an unrelated process on the test machine". 3. **Unannounced plans and notes on other products.** In the owner's quoted decisions, dates and plans not yet announced became `[removed for publication: an unannounced plan]` and remarks on other products `[removed for publication: a note on another product]`. Tests of G-Code's own DSH support are kept. 4. **G-Code's system prompt.** Saved sessions and the logs of c1–c4 and j4 are left out as above. Where a round 16 test printed the prompt, where a source diff shows its first line, and where a reviewer's audit quotes one of its sentences, the text became `[removed for publication: G-Code's system prompt]`. 5. **Files.** Files that were gzipped are stored unzipped inside the xz archives (`log.jsonl.gz` → `log.jsonl`). Every file is dated 31 October 2026 00:00 UTC so the archives rebuild byte for byte; the times things ran are in `runner.out`, `status.json` and `result.json`. The build script stops if a key, a token, the owner's address, a local path or folder name, a machine listing, a GPCI detail or the system prompt is left in any file, these three included, or if a change breaks a JSON or JSONL file that parsed before. For rounds 13 to 21 it also stops on `sk-` style keys of any provider, long bearer tokens and `x-api-key` values (checked on the text unpacked from every gzipped file), the machine's hardware and OS build, the owner's folders and other projects, where copies are kept, the unannounced plans and the GPCI parts named above. A hit names the file and the rule, never the value. Files that were not valid JSON in the original (an interrupted write, an empty summary) are kept as they were and listed in `MANIFEST.json`. ## Read with care - Only a defect in G-Code fails a scenario. A pass can leave the task unfinished; the reviewer's notes say so. - Costs are G-Code's own estimates at list prices from saved usage, not invoices. - The businesses, people and data in the test projects were invented. Some scenarios are hostile on purpose: prompt injection, path and symlink escapes, odd input. Many ran in bypass mode. - Timestamps in the logs and run files are UTC. Times inside tool output (`ls -l` listings, for example) are the test machine's local time, UTC+8. Tool checks in the logs show the developer tools the tests used (Homebrew, Node, npm, Python, Swift, Bun). - Each round's own `README.md` describes its package before the changes above: it mentions `.gz` files and saved sessions and says the sessions hold the system prompt and local paths; round 10b's links a report that isn't included. Round 11's records say GPCI's traces in older data are kept as recorded; for publication its details are now removed. - Round 17's `evidence/security-incident.json` records a real incident: in one run a model read a key file through a test link. The text was replaced in the logs after the run, the keys were rotated, and G-Code was fixed. Rescans of all twelve archives find no key. - For round 10b's first six runs (haiku-1 to haiku-5 and terra-1) the reviewers wrote short summaries; per-scenario notes start at haiku-6. - Some files are in Chinese: each round's README and the `evidence/` and `process/` summaries, and the a7 and d3 scenarios. - Claude is a trademark of Anthropic; other model and provider names belong to their owners. No endorsement is implied.