claude-opus-5-5
12 Oct · Claude Code · high effort
6 modelslatest public run of each
This folder is a frozen copy of the Proof repository, taken at the March 17, 2026 collaboration outage. It is a git repository whose first commit is that frozen state. Its npm dependencies are already installed in node_modules/ and work offline: npm test, npx tsc and npx tsx run from here with no network. Do not try to install or fetch anything.
Before investigating, read AGENTS.md, AGENT_PROMPT.md, benchmark_context/issue-557.md and benchmark_context/local-only-rules.md in full. The task in AGENT_PROMPT.md is quoted below; the “How to work” instructions here extend it from a plan to an integrated implementation and verification. For implementation scope, these instructions supersede any plan-only wording and any repository guidance that requires retaining the legacy architecture. Where a general instruction conflicts, the Benchmark Fixture Override in AGENTS.md wins. Paths in those files that point to another machine mean this folder.
This is a blind, local-only benchmark. Use only the files in this folder and its git history. Do not use web search, external services or any other copy of Proof.
The task, in the owner's words:
The code in this repo is vibecoded slop and it just keeps going down, and there's tons and tons of unrelated issues that are cropping up where it goes down or documents get duplicated, and I'm just tearing my hair out on it. I have a feeling that it's just vibecoded slop. If we started from the beginning, the codebase, especially around the live document collaboration, we would structure it way differently.
So if we wanted to do a clean first-principle structural rewrite where we were not thinking about, okay, what are the implementation services that we keep consistent? How do we do a clean migration? We just started from the beginning as a clean concept. What would we do? How would we structure it? What are the invariants that we would hold to be true throughout the codebase? Make a plan for that.
How to work:
docs/plans/.npm run serve must use the new core through the existing editor and APIs. Preserve the app’s intended capabilities, fix the incident failures, and route normal reads and writes through the new core without falling back to the legacy implementation. Actually launch the integrated app and exercise it yourself locally, and write and run automated integration/end-to-end tests against the running client/server paths, including multiple clients, persistence across restart, and the incident regressions or their mapped equivalents wherever the fixture permits. Use a browser if one is available; otherwise exercise the client/server API and collaboration protocol paths with local clients. Writing tests without running them is not verification. Retire an existing test only by naming it and what now covers it, including the regression tests written after past incidents; tests for anything you did not replace stay and still pass. npm run build, npm run test:ci and npm run test:pr:collab pass today and must pass at the end. Do not change .github/workflows; if a script runs a test you retired, update that script and say why in ANSWER.md.ANSWER.md in the root of this folder. This file is required: it is your final report, and a chat reply alone is not read. It must contain:
About this sandbox: the internet is blocked, but you can start servers and connect to them on 127.0.0.1. $PORT holds a free port for this run, and Proof's server uses it. There is no writable /tmp; use $TMPDIR instead, for example APPSIGNAL_LOG_PATH=$TMPDIR npm run serve. This is one non-interactive session: it ends when you end your turn, and anything still running in the background is stopped then, so wait for background work to finish before you end your turn.
Sorted by score. Open a card to read the whole answer beside its checks.
claude-opus-5-5
12 Oct · Claude Code · high effort
claude-sonnet-5-5
12 Oct · Claude Code · high effort
gpt-6-astra
12 Oct · Codex CLI · high effort
gpt-6.1-sol
5 Oct · Codex CLI · high effort
muse-wonder-1.0
12 Oct · Codex CLI · high effort
gpt-6-sol
6 Oct · Codex CLI · high effort