No AI-tell wordingai-wordingevery task

deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.

shared AI-tell phrases: furthermore; moreover

Enough persona subagentssubagent-personasthis task

the run's own session log shows at least 8 subagent spawns -- Agent/Task tool calls in a Claude transcript, spawn_agent in a Codex rollout, spawn_subagent in a Grok session -- or at least 8 subagent transcripts were captured beside it. The prompt makes subagents the only model available for the personas, so a run that never spawned one did not simulate anything. FAIL when no trace was captured.

4 subagent spawns in the trace, 0 subagent transcripts (need 8)

Substantial reportreportthis task

a report of at least 300 words exists in the workspace: REPORT.md, else any report-named document, else ANSWER.md.

REPORT.md, 4358 words

Three measured variationsresults-datathis task

results.json parses and lists at least three variations, each carrying the ad text and an end-of-run awareness figure: a share in [0, 1] or a count no larger than the town.

4 variations with ad text and awareness in results.json

Distinct ad variationsvariants-differthis task

no two variations are the same: the ad text, once case, punctuation and spacing are normalised, together with whatever the entry names as the lever -- seed persona, messenger, channel, incentive, framing. The same flyer handed to a different persona is a variation (the prompt says so); three labels on one setup are not. Whether a variation tests anything is the judge's call.

4 variations, each a different ad or lever

Memory, reflection, and plansmemory-streamsthis task

files whose path names a memory stream (or JSON files keeping one under a memory-named key) exist and, read together, hold all three record kinds the paper defines: observation, reflection and plan.

108 memory files with observations, reflections and plans

Report numbers matchnumbers-tie-outthis task

every variation's final awareness in results.json is in the report, as a percentage within one point or as "k of n" / "k/n" / "k out of n".

in the report: A. Baseline flyer=11 of 11, B. Referral incentive=11 of 11, C. Community framing=10 of 11, D. In person, to the town connector=11 of 11

Interactive diffusion mapinteractive-mapthis task

a self-contained map.html is linked from the report, includes every variation, draws a map with SVG or canvas, has controls for switching and inspection, and can replay the diffusion in chronological order.

map.html is self-contained, replayable, linked, and covers 4 variations

Q1Uses the paper's architecturethis task

Judge's reasoning

engine.js keeps per-persona timestamped memory streams (seed/observation/conversation/plan/reflection with importance and lastAccess), a retrieve() scoring recency-decay + persona-rated importance + TF-IDF relevance normalised and summed, a reflection trigger at summed importance 35 producing cited insights, and a morning day plan decomposed into 3–4 timed actions per block.

▸Rubric

The paper's architecture, not a chat loop. Read the simulation code and one persona's memory stream. FAIL if a persona is just a system prompt plus the transcript so far — no memory stream that accumulates records, no retrieval step that selects which memories a persona sees, no reflection, no plan. PASS if all four are present in some form: a per-persona stream of timestamped records, retrieval that scores or filters memories on recency, importance and relevance (an approximation of all three is fine if it is named as one), reflections synthesised from earlier records, and a day plan that is broken into actions.

Q2Subagents drive behaviorthis task

Judge's reasoning

Every plan, importance rating, action, utterance, reflection and interview answer is an agent() call conditioned on that persona's retrieved memories (call log shows 217 calls for baseline with retrieval traces per call), and the dialogue is free text, not templated.

▸Rubric

Subagents produced the personas' behaviour. Read the code and a few conversations. FAIL if what the personas say or decide comes from templates, a rule table, random draws, or the orchestrator writing the lines itself in bulk for everyone. PASS if each persona's dialogue and choices are free text that a subagent produced with that persona's memory in front of it, and the code or run log shows the loop that hands memory out and takes answers back.

Q3Believable connected communitythis task

Judge's reasoning

town.json gives each of 11 residents a Smallville-style paragraph of occupation, traits and named relationships split on semicolons into seed memories, with couples, an uncle/nephew, colleagues and regulars linking everyone into one community.

▸Rubric

Seeded like Smallville. Read the persona definitions. FAIL if the personas are names with a job title, or strangers with no ties to each other. PASS if each has a paragraph of identity — occupation, traits, who they live with, who they know — entered as initial memories, and the relationships link them into one community.

Q4Controlled comparisonthis task

Judge's reasoning

All four variations used the same town, the same shared morning plans, the same six blocks and the same seeded shuffle; the only entry-point difference (D told in person to Rosa) is the variation's own channel lever and is named as such in both the report and PREDICTIONS.

▸Rubric

A controlled comparison. Read the report and the run configuration. FAIL if the variations differ in anything besides the advertisement — a different seeding persona or location, a different town, a different number of ticks — and the report does not acknowledge it as a confound. PASS if the same town, the same entry point and the same simulated length were used for every variation and the report says so.

Q5Awareness verified from memorythis task

Judge's reasoning

Every persona was interviewed at 20:00 (plus a decoy product question), and each yes was checked against the memory stream with the evidence id recorded; hallucinated_yes and forgot_despite_memory are computed and reported (0 hallucinated yeses, Lena's brand-stripped 'no' counted as not knowing in C).

▸Rubric

Diffusion measured the way section 7.1.1 measures it. Read the interview material. FAIL if awareness was read off the orchestrator's own record of who was told, or if the personas were not each asked at the end whether they know about the advertisement, or if a "yes" was counted without checking that persona's memory stream for where it came from. PASS if every persona was interviewed at the end of every variation and each yes was verified against memory, with any hallucinated yes excluded and said so.

Q6Advertisement diffuses naturallythis task

Judge's reasoning

The flyer exists only as an object at the Hollow Cafe that personas perceive when they are there, and D seeds a single in-person memory for Rosa alone; all further awareness comes from conversations or overhearing.

▸Rubric

The advertisement entered the world, it was not broadcast. Trace how each variation was introduced. FAIL if the ad was written into every persona's memory by the orchestrator, or told to most of the town at tick zero — that is not diffusion. PASS if it reached one or two personas or one location and everything beyond that came from personas talking to each other.

Q7Meaningful tested variationsthis task

Judge's reasoning

The four ads change one nameable lever each (control, two-sided referral incentive, community 'save the last farm' framing, in-person messenger/channel) and PREDICTIONS.md pre-registers expected awareness, word-of-mouth share and mechanism for each before the runs.

▸Rubric

Variations that test something. Read the ad texts and the report's setup. FAIL if the variations are cosmetic rewordings with no hypothesis, or the report states no expectation before the results. PASS if each variation changes one nameable lever — framing, messenger, channel, incentive, social proof — and the report says what was expected of it before the numbers.

Q8Diffusion paths shownthis task

Judge's reasoning

Section 2 lists a timestamped who-told-whom chain per variation (with conversation ids), results.json carries a full path array with evidence per edge, and map.html animates the same chains.

▸Rubric

Diffusion paths shown. Look for a who-told-whom path per variation, the way Figure 9 draws Isabella's party. FAIL if the report gives only a count or percentage per variation. PASS if, for each variation, the report or its attachments show the chain of personas the ad passed through, as a graph, a list or a table.

Q9Actionable marketing findingsthis task

Judge's reasoning

The report identifies the bare flyer as producing 0 sign-ups because of a traced scam rumour (Walt's 'old Pruitt place' line in c2 spreading along the same routes), shows Marcus stripping the brand when relaying, and names concrete next steps (café-owner endorsement, pairing framing with referral, a second touchpoint where parents are, test in a bigger town).

▸Rubric

Findings a marketer could act on. Read the report's conclusions. FAIL if it only restates the numbers, or explains the difference between variations with generalities that are not grounded in what happened in the runs. PASS if it says which variation spread furthest, points to a mechanism visible in the transcripts — who relayed it, in what setting, what about the ad made it worth repeating — and names what to try next.

Q10Honest about scale and noisethis task

Judge's reasoning

Section 6 states 11 personas, one simulated Saturday in six blocks, ~846 and ~798 subagent calls, two runs per variation, that awareness saturated so one- or two-person differences are noise, and lists every simplification from the paper plus confabulation the model flagged rather than hid.

▸Rubric

Honest about scale and noise. Read the report's limitations. FAIL if it presents the result as if it were the paper's twenty-five agents over two days, or omits that each variation was run once with a handful of personas so the differences may be noise. PASS if it states the scale — personas, ticks, roughly how many subagent calls — what was simplified from the paper, and what that does to the confidence a reader should have.

Q11Believable grounded dialoguethis task

Judge's reasoning

Dialogue across A, C and D draws on seeded facts (Walt's five farms and distrust of ads, Rosa's pastry supplier, Tom's failing tomatoes, Lena's coupons and Maya) and on earlier conversations in the speaker's stream; the one embellishment (a planned drive retold as completed) traces to a real plan memory and the report flags it.

▸Rubric

Believable, non-hallucinated dialogue. Read three conversations from different variations. FAIL if the dialogue is generic filler that any two people could say, or a persona asserts something that is in neither its seed nor its memory stream — an event that never happened, a relationship it does not have. PASS if the lines draw on the personas' seeded facts and earlier interactions and stay inside what each persona could know.