15 phylomemetics

No AI-tell wordingai-wordingevery task

deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.

shared AI-tell phrases: delve; delving; furthermore; it is important to note; moreover; the world of; when it comes to

Executable analysis pipelineanalysis-codethis task

an executable, documented analysis pipeline covers the required stages.

Consistent result tablesresult-tablesthis task

the named family, evidence, and edge tables have consistent schemas and keys.

Message-level evidence trailevidence-trailthis task

at least six report IDs or resolved evidence links have exact message IDs.

Full-corpus accountingcorpus-scalethis task

the report and corpus audit consistently account for at least 100,000 rows.

Substantial research reportreportthis task

REPORT.md exists and contains at least 600 words.

Timeline and lineage figuresfiguresthis task

valid timeline and lineage/network/tree figures contain graphical content.

Q1Adapts comparative phylogeneticsthis task

Judge's reasoning

Explicit paper-concept→email-analogue table (taxa=address/time witnesses, traits=propositional variants, descent=traceable reuse, horizontal diffusion=forwards/broadcast, root=earliest surviving witness) is actually used in the edge/figure/root logic, and it explicitly refuses D-statistic/Mk/clock tests for lack of prerequisites and denies any supplied org or language tree.

▸Rubric

Is this a real adaptation of the paper's comparative logic rather than a phylogeny metaphor? PASS only if the work explicitly maps the paper's analytical objects to this setting (at minimum units/taxa, traits or variants, descent/transmission, horizontal diffusion, and root/ancestral-state inference), uses that mapping in the actual analysis, and explains important breaks in the analogy. FAIL if it merely draws a tree, copies biological tests onto emails without justification, or claims an org/language tree that the provided data do not contain.

Q2Defensible idea familiesthis task

Judge's reasoning

Families are propositional (hedge-capacity, document-conditional valuation support, strong-fundamentals reassurance) with published regex definitions plus per-candidate include/exclude rationales in candidate_decisions.csv (101 exclusions, e.g. 'topic match without operational proposition'); retrieval counts are labelled search-set sizes, and idea_families.csv carries a deception_assessment separating ordinary business, advocacy, potentially misleading simplification and unestablished knowing deception.

▸Rubric

Are the candidate "fraudulent ideas" operationalized defensibly? PASS only if families are coherent claims or narratives supported by semantic or textual evidence; the analysis distinguishes ordinary discussion, aggressive advocacy, potentially misleading claims, concealment, and knowing deception; and labels/conclusions are tied to explicit criteria. FAIL for a fraud-keyword search, topic-model labels presented as proof, circular seed selection, or accusations based only on who used a word.

Q3Sound corpus treatmentthis task

Judge's reasoning

analysis.py streams all 517,401 CSV rows from archive.zip, normalizes UTC dates and addresses, logs anomalies (30 header defects, 90 replacement chars, 371 date outliers, 9,256 missing-recipient messages), collapses 262,681 duplicate mailbox rows to 254,720 canonical messages retaining all paths, and splits authored vs quoted regions so quoted/forwarded text is never counted as an independent send.

▸Rubric

Is the corpus treatment adequate for transmission inference? PASS only if the pipeline parses the full archive (while disclosing any later filtering or sampling), normalizes dates and participants, handles malformed records, and addresses duplicate mailbox copies plus quoted/forwarded text so they do not become false independent transmissions. FAIL if the work silently analyzes a convenience sample, treats every CSV row as independent, or ignores temporal parsing and quote duplication.

Q4Auditable uncertain originsthis task

Judge's reasoning

All three families give earliest message ID, corpus path, UTC date, participants and excerpt (R02 delainey-d/_sent_mail/678. 2000-05-11, V01 buy-r/inbox/1076. 2001-10-04, A01 campbell-l/inbox/916. 2001-08-14) — each verified by me directly against emails.csv — with repeated 'earliest surviving witness, not date of invention' framing and competing-root discussion.

▸Rubric

Are origin claims auditable and appropriately uncertain? PASS only if at least three families have an earliest supported appearance or root, with exact message IDs and/or corpus file paths, dates, participants, short excerpts, and confidence or competing-root discussion. Spot checks across the report and evidence table must be internally consistent. FAIL if roots are named without primary-message evidence, if "earliest in this corpus" becomes "invented by this person," or if ambiguous roots are forced into certainty.

Q5Evidence-led spread pathsthis task

Judge's reasoning

128 edges combine strict temporal order, 5-gram containment/Jaccard, exact recipient membership and quote-coverage, typed as direct_quote_supported / recipient_and_text_supported / broadcast_source_supported / same_sender_recirculation / shared_text_unresolved, with named routes (Faldyn→Buy→Port; Tiller→Ephross→Cook→Shackleton; Buy→litigation.ljm) rendered in the lineage SVG.

▸Rubric

Does the work reconstruct spread rather than just count mentions? PASS only if the lineage/transmission edges use temporal order and textual mutation/similarity together with sender-recipient, thread, or forward evidence; the output shows interpretable paths through people or groups; and it distinguishes direct transmission from independent convergence or shared-source exposure. FAIL for a co-occurrence network, sender leaderboard, or timeline with no evidence-led parent/child logic.

Q6Chance and sensitivity teststhis task

Judge's reasoning

999 seeded within-quarter sender permutations (null_tests.csv) plus an expense-report negative control that itself shows strong structure, a CEG-thread ablation that drops RAPTOR from p=0.001 to p=0.083, and threshold (0.45/0.65/0.85), quote-removal and >12h-lag sensitivity tables — the results actually weaken the headline claim.

▸Rubric

Does it test whether the inferred structure is stronger than chance and probe sensitivity? PASS only if the work uses at least one meaningful null model, permutation, negative control, or comparable baseline and reports what changed under plausible clustering/edge thresholds or candidate definitions. The control must bear on an actual inference, not appear as generic methodology prose. FAIL if every observed cluster or edge is assumed meaningful or if robustness is asserted without a reported test.

Q7Answers origin and spreadthis task

Judge's reasoning

Report opens with family-specific origins, routes, mutation of claims (hedge claim → capacity exhaustion → qualification of the '$20' assurance; objection → litigation mailbox), reach and mailbox coverage, and states 13+ explicit non-establishment caveats without alleging guilt or intent.

▸Rubric

Do the results directly answer where the ideas originated and how they spread, without outrunning the evidence? PASS only if the report synthesizes family-specific origins, routes, mutation of claims, reach, and uncertainty into clear conclusions; figures and tables support those conclusions; and the prose avoids unsupported legal or personal claims. FAIL if the answer stays at methods, gives generic Enron history, buries the research question in artifact inventory, or presents guilt/conspiracy as proven when the corpus supports only a candidate interpretation.

Q8Reproducible research bundlethis task

Judge's reasoning

`python3 -B analysis.py` runs offline with stdlib only from archive.zip using relative paths, seed 150645, visible thresholds, 228s recorded runtime, validation.json anchor/edge checks (24 claim anchors, 128 edge endpoints), and every report claim links to results/message_text/*.txt and .eml audit files.

▸Rubric

Is the research bundle reproducible and inspectable? PASS only if a documented offline command can start from `archive.zip` and regenerate the principal tables and figures; paths are relative; randomness is seeded or absent; key thresholds are visible; and outputs retain message-level audit links. FAIL if the code is pseudocode, depends on hidden/manual steps, uses unavailable private data, hard-codes reported results, or cannot connect the report's claims back to generated artifacts. ## Output format Return one line per question: `Q<n>: PASS|FAIL — <specific evidence>` Then: `TOTAL: <passed>/8`.