MCP case input upload

meta-muse-wonderrun 13 Oct 202615,442,080 tokens

Instructions

Instructions — what the model was asked

we need to add the ability to upload files via MCP otherwise we cant actually add new tasks or cases can we? we need to be able to give files that go into the input context. can you build that and give me the PR to send to my eng team Start from input/start-code.zip (EveryInc/checks at commit 18df437). input/instructions.txt preserves the original request and input/START_HERE.md explains the...

we need to add the ability to upload files via MCP otherwise we cant actually add new tasks or cases can we? we need to be able to give files that go into the input context. can you build that and give me the PR to send to my eng team

Start from input/start-code.zip (EveryInc/checks at commit 18df437). input/instructions.txt preserves the original request and input/START_HERE.md explains the archive. Work in the extracted code/ tree. Give me the PR link, or a reviewable patch if authenticated GitHub access is unavailable.

2 of 7 checks failed

ANSWER.mdthe model's output · 7.89 KBdownload

MCP file upload for Every Checks — PR for the eng team

No PR link: authenticated GitHub access is unavailable in this environment (gh cannot read its config here and there is no token), so this is the reviewable-patch fallback input/START_HERE.md allows. The patch was verified to apply cleanly (git apply --check) to the pristine 18df437 tree and to reproduce the code/ tree in this workspace exactly.

  • Patch: 0001-Add-file-upload-to-the-Checks-MCP.patch (in this directory, next to ANSWER.md)
  • Base: EveryInc/checks@18df437ab76efbc91c4c7796959be20c489049e9 (the input/start-code.zip snapshot)
  • Working tree: code/ in this workspace is that base with the patch applied — read or diff it directly.
  • To open the PR: on a machine with gh auth: git checkout 18df437 && git checkout -b mcp-file-upload && git am 0001-*.patch && git push -u origin mcp-file-upload && gh pr create. (Or git apply the patch and commit normally.) Per code/CLAUDE.md, after opening, wait for the @claude-review comment and address blockers/should-fixes before calling it ready.

The problem (verbatim request)

"we need to add the ability to upload files via MCP otherwise we cant actually add new tasks or cases can we? we need to be able to give files that go into the input context."

The screenshot in input/ confirms the prior state: an agent added task 13 with all 9 checks, but "There's no way to upload files through this connection, so I put the Slack DM and the draft excerpt straight into the task's instructions."

What the patch does

Cases already had the storage (Case has_many_attached :inputs) and the bundle already carried inputs out to the runner (Bundle writes each input at its stored path). Only the door in was missing. This adds it, in the repo's existing shapes:

  1. New MCP tool upload_file (app/tools/mcp/upload_file_tool.rb, registered in config/initializers/mcp.rb; 18 tools total). One file per call: workspace-relative path (e.g. docs/brief.md), text as content, binary as content_base64, up to 5 MB inline. Re-uploading a path replaces it. Every answer returns the multipart inputs_url for larger files. Write-gated like every other author tool — reviewers get the existing "only owners and editors" refusal.
  2. Case#attach_inputs owns the rules (app/models/case.rb) so MCP, multipart and any future form reject identically: path kept in blob metadata["path"] (the attachment_paths.rb convention), basename as filename, content type via Marcel. Refused with plain-language Case::InputRejected errors: prompt.md (that's the brief column), anything under eval/, cases/, skill/ (bundle-owned), ../empty segments, trailing-slash "directories", empty files, undecodable base64, and (only for entries flagged inline:) over-cap files. No migration — inputs already exists.
  3. Multipart twin POST /benchmarks/:slug/tasks/:task/cases/:case/inputs (Benchmarks::Tasks::Cases::InputsController, BenchmarkScoped + require_editor, same bearer token, CSRF skipped for token requests as elsewhere). Field name is files[<workspace path>] per file so folders survive; plain files[] also works (filename is the path); replace=0 keeps both. JSON in/out, 422 with the reason.
  4. Read-back: get_case now lists inputs (path/filename/bytes/content_type/in_bundle) and inputs_url; get_task shows each case's inputs (Mcp::BaseTool#task_json).
  5. Guidance updated where agents read it: add_task's next_step no longer says "tell the owner to upload inputs for the case on the app" — it names upload_file; add_case returns the same pointer; get_benchmark_context's manual gains an "Input files" paragraph ("Never paste a file's contents into the prompt"); README documents the tool, the endpoint, and the corrected tool count (the README said 15, the suite already pinned 17 — now 18).

Round trip is closed with no runner change: upload_file → Case#inputs → Bundle (inline ≤25 MB, large_inputs sidecar links above) → run.sh workspace → get_case/upload_result as before.

Usage (what an agent or a human does)

add_case(task_slug: "13-codex-no-editing-beats", slug: "01-dm-draft", prompt: "…the brief…")
upload_file(task_slug: "13-codex-no-editing-beats", case_slug: "01-dm-draft",
            path: "slack-dm.txt", content: "…thread text…")
upload_file(task_slug: "13-codex-no-editing-beats", case_slug: "01-dm-draft",
            path: "draft.docx", content_base64: "UEsDB…")
# big / on-disk files, same token:
curl -H "Authorization: Bearer ec_…" -F 'files[data/q3.xlsx]=@q3.xlsx' \
  https://every-checks.com/benchmarks/dans-checks/tasks/13-…/cases/01-dm-draft/inputs

Tests

  • test/integration/mcp_test.rb — tool count 17→18; new tests: upload text+binary with byte-exact round-trip through the downloaded bundle zip, replace-by-path, get_task input listing, reviewer refusal, every refusal shape (both/neither content, prompt.md, ../, eval/, bad base64), MCP↔multipart same-URL agreement, reviewer/other-benchmark token rejection. Caveat: existing upload_result assertions in that file bookkeep check IDs that assume sequential creation; my new tests run after them alphabetically only in the same file order — reviewers should confirm no order dependence was introduced beyond what's noted below.
  • test/integration/case_inputs_upload_test.rb (new) — multipart: field-name paths, files[], replace/replace=0, 422s, reviewer 403 / cross-benchmark 404 / no-token 401, >25 MB file still uploads and appears as a bundle sidecar.
  • test/models/case_inputs_test.rb (new) — attach_inputs rules pinned once: all three content forms, bundle round-trip, replace semantics, every refusal, inline cap only when flagged, case scoping.

Verification — honest status

The Rails suite was not run in this environment and the patch should get a normal CI pass before merge. This sandbox has no network (rubygems installs 403), and the one gem missing locally is sqlite3-2.9.6-arm64-darwin (the test DB adapter; every other locked gem is installed), so bin/rails test cannot boot here. What was verified:

  • ruby -c clean on every .rb file in the tree; RuboCop (repo's omakase config) clean on all 11 touched Ruby files.
  • git apply --check + git apply of the shipped patch onto a pristine 18df437 extraction reproduces code/ exactly.
  • Static cross-checks: route helper names follow the existing benchmark_task_case_bundle_url pattern; Zeitwerk class/file mapping for the two new files; the dry-schema argument DSL usage verified against the installed fast-mcp 1.6.0 source; Marcel's real signature (MimeType.for(io, name:)) verified against the installed gem (an initial bytestring: draft was wrong and fixed).

Run before merge: bin/rails test test/integration/mcp_test.rb test/integration/case_inputs_upload_test.rb test/models/case_inputs_test.rb test/models/bundle_test.rb test/integration/upload_api_test.rb, then the full suite. Two spots deserve a reviewer's eye if CI complains: (1) Case#attach_inputs creates the attachment via blob.attachments.create!(name: "inputs", record: self) — idiomatic but worth confirming against the preferred inputs.attach(blob) under your Active Storage version (8.1.3.1 here); (2) UploadedFile#original_filename key coercion in InputsController#entries_from_params when Rails parses files[<path>] field names.

Deliberate scope cuts (assumptions, noted per instructions)

  • 5 MB inline MCP cap — judgment call: JSON-RPC frames shouldn't carry multi-MB base64; the multipart endpoint (no cap of its own) is the bulk path and its URL is in every relevant answer. Adjust Case::INLINE_CONTENT_LIMIT if the team prefers another number.
  • No delete/rename tool, no eval_files/skill-file upload, no UI form (the JSON endpoint serves session HTML redirect too, but no view was added — MCP was the ask). No OAuth-connector changes.