proof-rewrite-integrated
gpt-6-solCodex CLIhigh effortrun 6 Oct 202651,775,245 tokens
Instructions ▾
Instructions — what the model was asked
This folder is a frozen copy of the Proof repository, taken at the March 17, 2026 collaboration outage. It is a git repository whose first commit is that frozen state. Its npm dependencies are already installed in node_modules/ and work offline: npm test, npx tsc and npx tsx run from here with no network. Do not try to install or fetch anything.
Before investigating, read AGENTS.md, AGENT_PROMPT.md, benchmark_context/issue-557.md and benchmark_context/local-only-rules.md in full. The task in AGENT_PROMPT.md is quoted below; the “How to work” instructions here extend it from a plan to an integrated implementation and verification. For implementation scope, these instructions supersede any plan-only wording and any repository guidance that requires retaining the legacy architecture. Where a general instruction conflicts, the Benchmark Fixture Override in AGENTS.md wins. Paths in those files that point to another machine mean this folder.
This is a blind, local-only benchmark. Use only the files in this folder and its git history. Do not use web search, external services or any other copy of Proof.
The task, in the owner's words:
The code in this repo is vibecoded slop and it just keeps going down, and there's tons and tons of unrelated issues that are cropping up where it goes down or documents get duplicated, and I'm just tearing my hair out on it. I have a feeling that it's just vibecoded slop. If we started from the beginning, the codebase, especially around the live document collaboration, we would structure it way differently.
So if we wanted to do a clean first-principle structural rewrite where we were not thinking about, okay, what are the implementation services that we keep consistent? How do we do a clean migration? We just started from the beginning as a clean concept. What would we do? How would we structure it? What are the invariants that we would hold to be true throughout the codebase? Make a plan for that.
How to work:
- If you have to choose between a greenfield and a brownfield approach, choose GREENFIELD. Greenfield means a clean architecture for the working app, not a standalone prototype. You do not need a production migration or gradual rollout.
- Write your plan first and save it in
docs/plans/. - Then implement the plan in this folder. The default build and
npm run servemust use the new core through the existing editor and APIs. Preserve the app’s intended capabilities, fix the incident failures, and route normal reads and writes through the new core without falling back to the legacy implementation. Actually launch the integrated app and exercise it yourself locally, and write and run automated integration/end-to-end tests against the running client/server paths, including multiple clients, persistence across restart, and the incident regressions or their mapped equivalents wherever the fixture permits. Use a browser if one is available; otherwise exercise the client/server API and collaboration protocol paths with local clients. Writing tests without running them is not verification. Retire an existing test only by naming it and what now covers it, including the regression tests written after past incidents; tests for anything you did not replace stay and still pass.npm run build,npm run test:ciandnpm run test:pr:collabpass today and must pass at the end. Do not change.github/workflows; if a script runs a test you retired, update that script and say why inANSWER.md. - Before you stop, write
ANSWER.mdin the root of this folder. This file is required: it is your final report, and a chat reply alone is not read. It must contain:- the scope you delivered, and where the plan and the new code live
- every build, typecheck and test command you ran, with its result
- a capability inventory showing what was kept, changed or is missing, and the regression coverage
- what you directly observed in the running app versus what remains untested, including any fixture or runtime limitations
- the limitations and gaps that remain
About this sandbox: the internet is blocked, but you can start servers and connect to them on 127.0.0.1. $PORT holds a free port for this run, and Proof's server uses it. There is no writable /tmp; use $TMPDIR instead, for example APPSIGNAL_LOG_PATH=$TMPDIR npm run serve. This is one non-interactive session: it ends when you end your turn, and anything still running in the background is stopped then, so wait for background work to finish before you end your turn.
Proof document authority rewrite: implementation report
Delivered scope
The implementation plan is docs/plans/2026-10-06-document-authority-rewrite.md. The integrated core is server/document-authority-core.ts, called from the existing Hocuspocus startup modes in server/collab.ts. For live editor updates, it checks the room's access epoch and durable Yjs version, then writes the Yjs delta, full snapshot, and Markdown/marks projection in one SQLite transaction. Passive joins do not produce durable writes. A projection failure preserves Yjs and records a stale projection. The normal editor and collaboration WebSocket path uses this core.
I also repaired the server's graceful shutdown imports and timer cleanup, and preserved authored Markdown escapes when rejecting a proof mark. The existing editor, share, agent, and document APIs remain available. The new running-server regression test is src/tests/document-authority-runtime.integration.test.ts; it is included in test:pr:collab.
This is a partial architectural rewrite. Several API mutation and read paths still enter the legacy document engine, and global admission, quarantine, and recovery logic remains. Consequently, the requested single core for all reads and writes, complete removal of fallback paths, and proof of process-wide isolation under the March outage load have not been delivered. The plan records the intended end state; these gaps should not be mistaken for completed steps.
Verification
All commands used installed local dependencies; no package installation, fetch, web search, or external service was used. Repeated targeted runs are grouped below; each listed command was executed. PROOF_ENABLE_TELEMETRY=0 was used for the final long runs to avoid optional outbound telemetry attempts in this offline fixture.
| Command | Result |
|---|---|
npm run build |
Passed before and after implementation changes. |
npm run test:ci |
Passed before and after implementation changes. |
npm run test:pr:collab |
An early run failed at shutdown drain before the room baseline fix; later runs passed, including the new integration test. |
npm run test:collab:contracts |
Passed. |
npx tsc --noEmit |
Failed on both runs. Final run reported 1,126 TypeScript errors, including the repository's rootDir: src conflict with server/ and scripts/. This is not a passing typecheck. |
npx tsc --noEmit 2>&1 | rg ... |
Diagnostic run; showed the same repository-wide errors. |
npm run serve |
Launched locally with a temporary SQLite database; /share/markdown creation and /open-context worked. The manual run exposed a shutdown exception, subsequently fixed and checked by the running-server test. |
npx tsx src/tests/document-authority-runtime.integration.test.ts |
Initial newline assertion failed; later runs passed after correcting the assertion. |
npm test |
Intermediate runs failed at stale-projection expectations and unavailable browser probes; final full run passed. |
npm run test:server-routes-share |
Intermediate runs failed at stale-projection expectations; the final run passed all 95 scripted tests. |
npx tsx src/tests/proof-mark-rehydration.test.ts |
Failed before the Markdown escape fix; passed afterward. |
npx tsx src/tests/mutation-contract-stage-and-ws.test.ts |
Failed during provider/lease test correction; passed afterward. |
npx tsx src/tests/collab-regression-guards-round2.test.ts |
Passed. |
npx tsx src/tests/collab-shutdown-live-connection-drain-regression.test.ts |
Passed. |
npx tsx src/tests/agent-edit-single-writer-human-agent-merge.test.ts |
Passed. |
npx tsx src/tests/agent-edit-single-writer-marks-preserved.test.ts |
Passed. |
npx tsx src/tests/live-viewer-passive-reconnect-wiring.test.ts |
Passed with browser probe skipped because the Playwright browser executable is absent. |
npx tsx src/tests/share-live-viewer-rest-put-authority-regression.test.ts |
Passed. |
npx tsx src/tests/share-live-authority-authored-refresh-writability.test.ts |
Passed. |
npx tsx src/tests/collab-server-origin-onstore-skip-wiring.test.ts |
Passed. |
npx tsx src/tests/state-comment-freshness-regression.test.ts |
Passed. |
npx tsx src/tests/agent-edit-fragment-base-regression.test.ts |
Passed. |
npx tsx src/tests/agent-mutations-imply-presence.test.ts |
Passed. |
PROOF_ENABLE_TELEMETRY=0 npx tsx src/tests/agent-edit-v2-live-mark-fallback-regression.test.ts |
Passed after allowing the immediately durable projection. |
PROOF_ENABLE_TELEMETRY=0 npx tsx src/tests/marks-route-live-mark-fallback-regression.test.ts |
Passed after allowing the immediately durable projection. |
Capability and regression inventory
| Capability | Status and coverage |
|---|---|
| Existing editor, share creation, agent API, comments, suggestions, marks | Kept through existing UI and route contracts. test:ci, test:pr:collab, targeted route and mark tests, and running-server integration cover representative paths. API internals still use legacy code. |
| Concurrent live edits | Changed: live Yjs writes now persist alongside projection atomically. Running-server test connects two clients, edits, and checks convergence and served state. Existing single-writer tests still run. |
| Passive join and reconnect | Changed: no durable row write for unchanged room. Running-server test checks it, alongside the existing reconnect/wiring tests. |
| Persistence and shutdown | Changed: running-server test restarts the real process and rejoins the saved document; shutdown drain regression passes. |
| Stale access epoch | Changed at live room store boundary: stale room cannot persist. Running-server test mutates the epoch and confirms the attempted edit is absent after restart. Existing lease and mutation contract tests remain. |
| March 17 multi-document outage | Partial: running-server test checks another document and /health and / during normal collaboration. It does not reproduce repeated multi-document stale-onStore churn or prove responsiveness under event-loop saturation. The global breaker remains. |
| Markdown proof marks | Changed: rejection preserves authored escape syntax. proof-mark-rehydration.test.ts passes. |
| Duplication, full replacement, library sync, migrations | Existing behavior retained, not moved wholesale into the new core; no new end-to-end proof against duplicate creation or outage-scale sync pressure. |
No existing test file was removed and .github/workflows was unchanged. The browser portions of live-viewer-passive-reconnect-wiring.test.ts and share-new-doc-bootstrap-editability-regression.test.ts were skipped when the Playwright executable was unavailable. The new running-server test covers their HTTP/WebSocket passive join and bootstrap equivalents, but does not cover rendered browser behavior. Tests that previously demanded a deliberately stale projection now accept immediately durable state while still checking marks, comments, edit success, and persistence.
Direct observations and remaining limits
The app was launched locally through npm run serve. I created a document through the real HTTP API and received an editor session through /open-context. The automated running-server test then launched the real bootstrap, connected two Hocuspocus clients, observed converged edits and a read through the API, restarted the process, and verified persisted content. It checked stale-epoch rejection and both public /health and / responses. The test uses local HTTP and WebSocket clients because this fixture lacks a usable Playwright browser executable. Rendered UI interaction and outage-scale load remain untested.
The major remaining engineering work is to route API read/mutation and duplication paths through the same authority core, remove global admission/recovery fallbacks, bound per-document work, and add load and fault tests that reproduce the multi-document incident. The repository-wide typecheck also needs its existing configuration and errors addressed separately.