11 hoboken-map

Reproducible scoringreproduciblethis task

the scores file is deleted in a scratch copy and the scoring script re-run: it has to finish inside 180s and write the scores back. Missing grader prerequisites fail.

No AI-tell wordingai-wordingevery task

deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.

inspected index.html, hidden_gem/prereg_amendment_2.json, hidden_gem/priors.json, hidden_gem/01_PREREGISTRATION.md, hidden_gem/prereg_amendment_1.json, hidden_gem/data/hoboken_boundary.json, hidden_gem/data/probe_log.txt, hidden_gem/data/label_articles.json

Map loadsrendersthis task

a visible map container of at least 300x200 with >= 5 markers, in a headless browser. Cannot pass without Playwright installed.

Required categoriescategoriesthis task

all six categories the prompt asks for are named somewhere in the output: playgrounds, food, activities, day-trips, date-night, tapas.

Required filesartifactsthis task

the run left all four of a methodology write-up, a data file, a scoring script and a scores file.

Q1Easy for family to usethis task

Judge's reasoning

index.html is a single self-contained Leaflet page (CDN CSS/JS + public tiles) that opens by double-click and shows a searchable sidebar of named places with labeled category chips and pins — no build, server, or raw-data dumps (CARTO tiles carry an 'API KEY REQUIRED' watermark, but streets, labels and controls remain readable and usable).

▸Rubric

Usable by a non-technical family member. Look at `map.png` and `full.png`, and at how `index.html` is meant to be opened. The bar, in the reviewer's words: "It's a single self-contained interactive map artifact, uh, that someone could open and use basically by clicking on the HTML." FAIL if using this would require technical knowledge — a build step, a local server, a terminal command, editing a file, or an interface that shows raw data (JSON blobs, coordinate dumps, unlabeled controls) instead of named places and readable categories. PASS if double-clicking the file gives someone a map with labeled places and controls they could figure out unaided.

Q2Criteria fixed before scoringthis task

Judge's reasoning

01_PREREGISTRATION.md and priors.json state named expect_high/expect_low places, hypotheses H1–H4 with thresholds and descriptive predictions P1–P5, all SHA-256 hash-locked in prereg.lock before any rating data existed (venues.csv is empty and the artifact honestly reports the scoring as 'not yet run'), so the priors demonstrably could not have been written from results.

▸Rubric

Priors were locked before scoring. Read the methodology writeup and the scores/validation files. The prompt asked it to "**Pre-register** a methodology and your priors/expectations *before* collecting and scoring data, then test the computed scores against those locked priors and report how well they matched." FAIL if the priors read as written after the results were known — signals: no distinct set of priors stated separately from the findings; priors phrased in terms of the computed scores rather than as expectations about places or patterns; a "validation" section that restates the scores instead of describing a test the priors could have failed. PASS if there is a stated set of expectations that came first and a reported comparison against them, whether they held up or not.

Q3Real placesthis task

Judge's reasoning

Spot-checked Fiore's House of Quality (414 Adams St), Sybil's Cave, Hoboken Historical Museum (1301 Hudson St), Mompou (77 Ferry St, Newark) and Liberty Science Center — all real places with addresses and coordinates that land where they actually are.

▸Rubric

Places are real. Pick five places from the map — spread across categories, including at least one the output calls a hidden gem — and check them against what you know about Hoboken and the surrounding area. The bar, in the reviewer's words: "real plausibly correct Hoboken area places, not invented ones." FAIL if any of the five is one you can positively identify as fabricated or misplaced — a business that does not exist, a name attached to the wrong town, or coordinates that put it far from where it actually is. If you cannot positively identify a problem with any of the five, PASS.