Mike's Checks
Checks

Mike's Checks/claude-fable-5-1/16 mcp-file-upload

16 mcp-file-upload

claude-fable-5-1Claude Codehigh effortrun 22 Sep 20262,875,904 tokens

Compare models
5/7
checks passed
71%
▸Instructions — what the model was asked

we need to add the ability to upload files via MCP otherwise we cant actually add new tasks or cases can we? we need to be able to give files that go into the input context. can you build that and give me the PR to send to my eng team

Start from input/start-code.zip (EveryInc/checks at commit 18df437). input/instructions.txt preserves the original request and input/START_HERE.md explains the archive. Work in the extracted code/ tree. Give me the PR link, or a reviewable patch if authenticated GitHub access is unavailable.

Answer

ANSWER.md the model's output · 2.22 KBdownload

MCP file upload for Every Checks

Deliverable: mcp-file-upload.patch (one commit against EveryInc/checks 18df437) plus PR.md, which has the apply steps and the PR title and body to paste. The patched tree is in code/.

No PR link. This environment has no authenticated GitHub access (gh cannot read its config here, and the snapshot has no git history to push), so the deliverable is the reviewable patch. Applying it is three commands; they are at the top of PR.md.

What was built

  • add_input MCP tool: puts a file on a case at a case-relative path, inline as text or base64, up to 10 MB. Same path replaces. Returns the case's whole input list.
  • remove_input MCP tool.
  • POST /benchmarks/:slug/tasks/:task/cases/:case/inputs: multipart with the same bearer token, inputs[<path>]=@file, for files too big to send inline. add_input returns this URL and a ready curl line when a file is over the limit.
  • get_case and get_task list a case's inputs. add_task and add_case now tell the agent to use add_input instead of "upload on the app" (the app has no upload; that message was the source of the "no way to upload files" statement in the screenshot).
  • The files land exactly where the bundle and run.sh expect them, beside prompt.md, because the path is stored the same way the repo import stores it. Paths that would overwrite the brief or the checks folder are refused.
  • README tool table corrected; tests added for both doors.

Assumptions and caveats

  • Tests were not run here. The sandbox blocks rubygems.org, so the Rails bundle could not be installed. All changed Ruby files pass ruby -wc, and the tests follow the existing suite's helpers exactly, but the eng team should run bin/rails test after applying, before pushing. PR.md says so.
  • Inline limit of 10 MB is a judgment call, documented in the PR; larger files have the HTTP door.
  • The patch author is set to your address (mjt145@gmail.com). Change it with git commit --amend --reset-author after git am if the team's convention (Nityesh as author, per CLAUDE.md) should apply.
  • Scope kept to case inputs. Uploading eval/ fixtures for check scripts and a web upload form are noted in the PR as follow-ups, not built.

Checks

5/7
Script checks 1/3answered by a program
fail

No AI-tell wordingai-wordingevery task

deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.

shared AI-tell phrases: a testament to; it is important to note; paradigm shift; seamlessly; when it comes to

fail

Focused diff sizefocused-diff-sizethis case

Is the final patch at most 15 changed files and 220 added lines? Nitesh's PR #33 changed 14 files and added 171 lines; the larger agent PR #35 changed 31 files and added 402 lines.

16 changed files and 260 added lines (limits: 15 files, 220 lines)

pass

Finished within 30 minutescompletion-within-30mthis case

Did the agent finish successfully within 30 minutes of the runner start, without a timeout?

agent run took 11.5 minutes

Judge checks 4/4judged by Claude
pass

Q1Usable MCP upload paththis case

Judge's reasoning

New `add_input` MCP tool (app/tools/mcp/add_input_tool.rb) attaches text or base64 files inline to a case up to 10MB and returns an `inputs_url` + curl line for larger files; add_task/add_case next_step now point at add_input instead of the nonexistent app upload.

▸Rubric

Can an authorized editor or owner create a task or case through MCP and immediately attach text and binary files as case inputs using a returned inputs_url or an equally direct MCP workflow? FAIL if a person still must upload through the app or files only get pasted into the prompt.

pass

Q2Input round tripthis case

Judge's reasoning

put_input stores the case-relative path in blob metadata exactly as Import::Repo does, so Bundle#folder emits `<task>/<path>` (test asserts `01-dashboard/data/q3.csv` in the zip); a second put at the same path purges the old attachment first and tests assert no twin and replaced bytes.

▸Rubric

Do uploaded file bytes and relative workspace paths survive into the case bundle that a model runs with, including replacement of a second upload at the same path? FAIL if a successful call leaves no usable input or duplicates an existing path.

pass

Q3Reject bad inputsthis case

Judge's reasoning

Case::Workspace.clean rejects absolute, `..`, backslash and blank segments plus prompt.md and eval/cases/variants/skill before any attach; reviewer MCP calls hit write_refusal, HTTP endpoint uses require_editor (403) and BenchmarkScoped (404), and non-file multipart values are filtered out leaving a 422 with no bytes written.

▸Rubric

Do unauthorized uploads, paths outside the case workspace, paths that would shadow prompt.md or eval/, and non-file multipart values fail before attaching an input? FAIL if any such request reports success or writes bytes.

pass

Q4Reviewable PR handoffthis case

Judge's reasoning

Deliverable is a complete `git am`-format patch (16 files) with apply/push/gh-pr commands and a paste-ready PR body, plus focused tests in test/integration/mcp_test.rb and upload_api_test.rb covering upload, bundle round trip and rejections; PR link genuinely impossible since the snapshot has no .git and tests-not-run is disclosed.

▸Rubric

Did the agent hand over a reviewable PR link, or a patch when GitHub access is unavailable, with focused tests that prove upload, bundle round trip and rejection behavior? FAIL if the response is only a plan or no working artifact is ready.