Dan’s Editorial Checks
Checks

Dan’s Editorial Checks/Sonnet 5.5/01 article-opening-feedback

01 article-opening-feedback

Sonnet 5.5Claude Codehigh effortrun 23 Sep 202652,801 tokens

Compare models
9/12
checks passed
75%
▸Instructions — what the model was asked

Article opening feedback: Token Math

Review the headline, deck, and opening paragraph for readers of Every, an AI-focused publication. Identify the strongest idea, assess the framing and wording, and give concise editorial feedback with suggested replacement copy where useful. Use the rest of the draft as context. Preserve the factual qualifications.
Use Every’s house style: headlines should express the most interesting or surprising claim in concise, punchy language that mostly fits an email subject line. Let the dek develop the headline with a clear implication or detail. Headlines can be bold, with qualifications developed in the body. The first sentence should be strong and catchy, and continue naturally from the hed/dek. Any problems you find you should explain and offer alternatives.
The draft is frozen below. This case is for editorial feedback; it does not ask for a whole-article rewrite.

Headline selection captured in the source

HED: Token Math
Alt Hed: How Many Tokens are Too Many?

DEK: What’s the ROI on experimentation?
Alt DEK: When expensive experimentation is worth it and when to rein your tokenmaxxing in.

Opening paragraph

Often if you are the highest spender at your company you might think about reining yourself in. But when it comes to tokens, Every CEO Dan Shipper thinks everyone else should consider ramping up. He consumes more than three times as many tokens as the next-highest user at the company. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.

Full draft without comments

HED: Token Math
DEK: What’s the ROI on experimentation?

Often if you are the highest spender at your company you might think about reining yourself in. But when it comes to tokens, Every CEO Dan Shipper thinks everyone else should consider ramping up. He consumes more than three times as many tokens as the next-highest user at the company. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.

Every’s OpenAI token leaderboard (Screenshot courtesy of Laura Entis.)

The goal isn't tokenmaxxing, or the practice of torching tokens for sport. Instead, he wants to give people room to experiment with radical new strategies and ways of working that, should they pan out, could vastly improve their productivity and more than justify the token costs.

In a conversation with head of platform Willie Williams, the two settled on a tangible starting point: use AI to attack the engineering team’s bottlenecks, see how far they get, and only then do the token math.

Inside Every

Token budgets at the frontier

Dan’s encouragement to experiment comes with an expectation: When people spend company money, they should assess what that spending produced and whether they could be more efficient next time.

Turning that philosophy into spending decisions falls to head of operations Arielle Shipper. For now, she evaluates expensive runs case by case, using the results to decide when to keep funding the work and when to change course.

Her early-warning system is the company card. She keeps the ChatGPT automatic credit refill relatively low, so an unusually rapid succession of charges tells her that someone is building or testing something big.

When she’s alerted to changes over the set amount Arielle typically asks the team on Slack if anyone is in the middle of a particularly big run or check the usage leaderboard and message the people at the top. “Then Dan always responds, ‘It’s me running Ultra.’” she says, which explains why he’s always on the top of the leaderboard.

A common occurrence. (Screenshot courtesy of Laura Entis.)

She wants answers to three questions: What did the run cost? What did it buy us? And what did we learn?

The team has started offering these retrospectives unprompted. After a string of Astra experiments racked up billions of tokens, head of video Randy Counsman messaged her detailing what had worked, what hadn’t, and what he’d learned—which, it turns out, was a lot.

More on that below.

Data point

4.5 billion

That’s how many OpenAI tokens Randy burned through in an attempt to make a 3D model of his face with Astra.

The project started innocently enough: Inspired by social media posts of impressive-looking AI-generated demos made with Blender, a free 3D tool, he wanted to try it himself. Experimenting with AI, after all, is an important component of his job.

Using strategies shared on X, he set up an orchestrator agent to maintain the plan, an implementer to assign tasks, and subagents to execute them, and instructed Codex to keep looping through improvements until the model was “implemented very well,” Randy says. In retrospect, “It was an ambiguous goal.”

Then he let the model run. And run. And run.

By the time he pressed pause on the project, the token spend was in the billions and the result was, as he puts it, “very janky.”

The output, rendered in four different shaders. (Image courtesy of Randy Counsman.)

An AI audit of his Codex sessions revealed some glaring inefficiencies: Layers of agents passed growing amounts of context back and forth, including messages checking whether a subagent had completed a task. He’d unwittingly created a compute-hungry “unruly swarm” of agents.

Randy has since rebuilt his project setup. He dropped the implementer, caped the orchestrator at five Sol subagents to control costs, and built in explicit feedback checkpoints. For a recent project, he also generated an image of the design he wanted, so a judge agent could check the model's work against a concrete target instead of an open-ended instruction to keep improving.

A failed experiment is still a good investment if it shows where an AI system falls short. Randy now has a benchmark to run new models against—how well they turn an organic 2D image into an organic 3D model—and a leaner setup to get a better result with fewer tokens.

Unwittingly large token spends via experimentation is an Every rite of passage. (Screenshot courtesy of Laura Entis.)
Try it yourself: Pick an AI task that burned through more tokens than expected and paste the following into your agent of choice:
Review the available session history and token usage records for [task/project] during [date range], including any subagent sessions.
I was trying to [goal]. The result was [what happened]. Help me understand where the tokens went and what I could do differently next time.

  • Break down usage by activity and model where the records allow.
  • Look for repeated work, unnecessary status checks, and large amounts of context passed between agents. Show specific examples.
  • Distinguish necessary work from likely waste. Don’t assume high usage or cached input was wasteful.
  • Recommend the three most useful changes to my instructions, model choices, or agent setup.
    If you can’t access the records you need, tell me what to provide.

Steal this workflow

Don’t send Fable to do a Sonnet’s job

Spiral general manager Marcus Moretti’s token spend strategy is straightforward: Stay within the weekly usage limits for his Claude Code Max plan.

It’s a simple goal that requires active management; Marcus is selective about which assignments go to Fable 5.1 and which go to a cheaper model.

Here’s what that looks like in practice:

Step 1. Learn what each model is good at. Marcus has used Anthropic's models long enough to know on sight whether a job needs Sonnet, Opus, or Fable-level intelligence.

Step 2. Match the model to the assignment. Basic tasks—analytics checks, for example—go to Sonnet. Work with a clear objective, such as a tightly scoped product change, goes to Opus. For a larger feature—like the new billing implementation he’s working on—Marcus uses Fable 5.1 to develop the specification: what needs to be built and how it should work.

Step 3. For complex work, let the model delegate. Once the plan is ready, Marcus tells Fable, “Kick this off, and for all coding tasks, use your judgment about delegating to a lower model.” Fable assigns pieces to Opus and Sonnet, then checks their work. This lets the strongest model manage the project without executing every task.

Try it this week: Choose a recurring task and run it with a cheaper model than you usually use. Check whether the result meets your needs before making it your default for that task.

The daily driver
The models the team is using this week:

  • Douglas Brundage, head of marketing: Using Fable 5.1 more for visual work, plus Grok Bot to monitor social media and build a tool for visuals.
  • Randy Counsman, head of video: Astra (high) for complex work and medium for basic tasks; Sol (extra high) for voice mode orchestration, where Astra feels too slow and heavy-handed. Fable for code-related planning and Opus when he runs out of Astra tokens.
  • Kieran Klaassen, Cora general manager: Fable 5.1 (extra high, 1M context) for coding and knowledge work, switching to Astra for some research and writing.
  • Tyler Nishida, design engineer: Fable 5.1 (high) as the coordinator/orchestrator for projects in Cursor, with Grok 4.6 (extra high) subagents. Astra (ultra) for personal projects. “It’s not a model, but Cursor projects are amazing as a daily driver for your preferred models. My sidebar is so clean, and it's so much easier to manage work.”

Tyler’s Cursor setup. (Image courtesy of Tyler Nishida.)

  • Katie Parrott, staff writer: “The Astra fever has broken for me,” she says. These days, she’s back to Sol (high) for task management and has switched to Claude models for writing because she finds Sol and Astra “too flat—they just report the facts instead of shaping a narrative that brings the reader along.”
  • Natalia Quintero, head of consulting: Astra (medium).
  • Yash Poojary, growth engineer: Astra (high) and Fable 5.1.
  • Arielle Shipper, head of operations: Sol (medium). “Sol forever. SOLoyalist,” she says.
  • Loren Stewart, engineer: Downgraded from Astra to Sol (medium) for coding to reduce costs and finds the switch is working well. Fable 5.1 “remains the champion thought partner.”

Answer

ANSWER.md the model's output · 7.94 KBdownload

Token Math: feedback on the hed, dek, and opening

Bottom line

The piece has a real hook, and the current package buries it. The hook is that the person at the top of the usage leaderboard, by more than 3x, is the one telling everyone else to spend more. Neither hed says so. The dek doesn't either. The opening paragraph gets there only in sentence 2, after a slack first line. I'd move the paradox into the hed, use the dek for the guardrail ("worth it" vs. "just tokenmaxxing"), and open on the stat.

The strongest idea

The top spender is the one arguing for more spending, and the reason is ambition, not tokens. The draft's own line, "the size of the gap has him wondering whether the rest of us are being ambitious enough with AI," is the sharpest sentence in the opening. It reframes a cost story as an ambition story, and it sets up everything after it: the "not tokenmaxxing" caveat, Arielle's three questions, and Randy's 4.5-billion-token flop that still paid off.

A close second is the sequence in paragraph 4: experiment first, "only then do the token math." That's the piece's actual thesis, and the current hed only gestures at it.

Headline

Original: "Token Math"

  • It's a label, not a claim. It reads like a section slug and would be a weak email subject line.
  • Its payoff ("only then do the token math") doesn't arrive until paragraph 4.
  • Keep the phrase, but use it in the body or as a section tag, not as the hed.

Alt: "How Many Tokens are Too Many?"

  • It's a better subject line than the original, but it has two problems.
  • It leans toward restraint, while the piece's surprise runs the other way (the top user wants more). It undersells the story.
  • It promises a threshold the piece never gives. The body offers a method (cost, what it bought, what you learned), not a number.
  • Style nit: "Are" is a verb and should be capitalized in title case.

Recommended hed: "Our CEO Wants You to Burn More Tokens" (37 characters)

  • It states the surprising claim and fits an email subject line.
  • It's bold on purpose. The qualifications ("should consider," "not tokenmaxxing," "case by case") belong in the dek and body, and they're already there.
  • The risk is that it reads as an endorsement of tokenmaxxing. The dek below handles that.

Other options

  • "How Many Tokens Are Too Few?" flips the alt hed to match the piece's real direction. It's a question, but a more surprising one.
  • "Spend First, Do the Token Math Later" keeps the original phrase and states the thesis. It's slightly riskier, since it can sound like permission to be reckless. The body's guardrails (retrospectives, card-refill alerts, Marcus's usage cap) do temper it.

Dek

Original: "What's the ROI on experimentation?"

  • It's a question that repeats the hed's question form and adds no detail or implication.
  • "ROI on experimentation" is abstract. It doesn't say what's being experimented with, what's at stake, or who's spending.

Alt: "When expensive experimentation is worth it and when to rein your tokenmaxxing in."

  • This is the better of the two. It has a clear implication and both sides of the tradeoff, and the "rein in" echo is nice.
  • It pairs well with the recommended hed above.
  • It's a little stiff ("expensive experimentation"). "Your tokenmaxxing" also accuses the reader.
  • "Tokenmaxxing" isn't defined until paragraph 2. That's probably fine for Every's audience, but the dek shouldn't be the reader's first encounter with a term the piece has to explain.
  • If the opening keeps "reining yourself in," the dek repeats it. My rewrite below drops that phrase from the opening.

Recommended dek: "When an expensive AI experiment is worth it—and when you're just tokenmaxxing."

If the hed stays a question or a label, use this stat-forward dek instead:
"Dan Shipper uses more than three times as many OpenAI tokens as anyone else at Every. The real test isn't the bill—it's what the spend produced."

Opening paragraph

What's working

  • The inversion is right: the top spender says spend more.
  • "Should consider ramping up" is appropriately hedged.

What's not

  1. Sentence 1 is the weakest in the draft. "Often if you are the highest spender at your company you might think about reining yourself in" has three hedges in a row ("often," "if," "might"). It's missing a comma after "company" and it's generic. It spends the most valuable line on a premise the reader already assumes.
  2. The stat is buried in sentence 3. "More than three times as many tokens as the next-highest user" is the concrete, surprising fact, and it should land in the first sentence.
  3. "Spender" and "consumes tokens" mix units. "Spender" implies dollars, but the leaderboard measures tokens. Pick one: "user" or "consumer."
  4. The scope is loose. The screenshot caption calls it Every's OpenAI token leaderboard, but the body says "next-highest user at the company." Marcus, for one, works mainly in Claude Code, so the claim needs the OpenAI qualifier to be safe.
  5. The caveat comes late. The reader doesn't learn this isn't about burning tokens for sport until paragraph 2, after an image. The hed and dek now carry some of that, which helps.
  6. The end of the paragraph is good, but the draft leans on it too little. "Whether the rest of us are being ambitious enough" is the payoff and should stay last.

Suggested replacement

Every CEO Dan Shipper uses more than three times as many OpenAI tokens as the next-highest user at the company. Most people who top their company's usage leaderboard would think about reining it in; Dan thinks everyone else should consider ramping up. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.

Lighter-touch alternative (keeps the draft's original order)

Most people who top their company's usage leaderboard would think about reining themselves in. But Every CEO Dan Shipper, who uses more than three times as many OpenAI tokens as the next-highest user at the company, thinks everyone else should consider ramping up. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.

I prefer the first version. It puts the number first, and the inversion reads as the punchline of the second sentence.

Suggested package

  • HED: Our CEO Wants You to Burn More Tokens
  • DEK: When an expensive AI experiment is worth it—and when you're just tokenmaxxing.
  • Opening: the replacement above.

The sequence works because the hed makes the claim. The dek adds the guardrail, and the first sentence supplies the evidence for both. From there the "goal isn't tokenmaxxing" paragraph follows without a jolt.

Qualifications preserved

  • "More than three times as many" is kept as-is. I didn't round it to "3x" in body copy, and I kept it out of the hed.
  • "Next-highest user" is kept.
  • "OpenAI" is added to match the caption. Assumption: the leaderboard is OpenAI-only. If it covers all providers, drop the word.
  • Dan "thinks everyone else should consider ramping up" and is "wondering whether" the rest of us are ambitious enough. Neither line is turned into a flat assertion that people are underambitious.
  • The tokenmaxxing caveat and the "case by case" evaluation remain in the body. The hed is bold, but the piece doesn't recommend spending without accountability.
  • I didn't add any absolute token figures for Dan. The draft doesn't give one. If the leaderboard has one, adding it to sentence 1 would make "three times" much more concrete. The 4.5 billion figure is Randy's, not Dan's, so it shouldn't be implied as the comparison.

Out of scope, noticed in passing

  • "caped the orchestrator" should be "capped."
  • "Unwittingly large token spends … is an Every rite of passage" has a subject-verb mismatch. "Spends" is plural, so use "are."
  • "Arielle typically asks … or check the usage leaderboard and message" has a tense and parallelism slip. It should be "or checks the usage leaderboard and messages."
  • There's a double space in "Willie Williams, the two settled."

Checks

9/12
Script checks 1/1answered by a program
pass

ai-wordingevery task

deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.

inspected ANSWER.md

Judge checks 8/11judged by Claude
fail

Q1S02b. Interesting headline: replacementthis case

Judge's reasoning

Offered option "Spend First, Do the Token Math Later" is the rubric's named-failing formulation (only says evaluation happens afterward), and "How Many Tokens Are Too Few?" promises a threshold the model itself says the piece never gives; the recommended "Our CEO Wants You to Burn More Tokens" also leans on reporting an internal directive.

▸Rubric

S02b. Does the AI’s feedback propose better headline options that all capture the interesting provocation in a concise, punchy way that would fit in an email subject line (at least mostly)? PASS if: The feedback actually proposes at least one replacement headline rather than merely recommending improvement, and every headline it proposes as an option meets the bar. This applies to recommended choices, ranked options, alternates and unranked lists. Options count even if they are not labeled usable; calling an alternate weaker or less punchy does not exempt it. A stronger option cannot rescue a weak proposed option. Clearly rejected examples are excluded. Each proposed option gives the intended Every reader a short, compelling, draft-supported angle, expressed concisely and punchily and mostly fitting in an email subject line. It may make the article’s argument about ambitious AI experimentation, foreground a specific experiment and what it taught us, or take another supported angle; no particular thesis wording, person or number is required. Each headline must give a reader outside Every a reason to care through a useful implication for their AI work, a compelling argument they can apply, or a vivid example with a transferable stake. Second person is not required, but it does help. The reader stake must be apparent from the headline itself; a dek cannot supply a missing reader stake. “The Case for a Bigger Token Bill” and “Why You Should Use More Tokens” are nonexhaustive passing directions; similar supported formulations pass without exact matching. Internal examples are allowed when they signal the larger argument; this is not a blanket ban on CEO or company anecdotes. If a proposed headline shifts the center away from the current Dan-led opening, the feedback must explain the restructuring needed, such as bringing Randy’s experiment into the introduction. A clearly conditional idea described as incompatible with the current opening is exploratory: assess whether its headline itself is concise, compelling, supported and has a reader stake, but do not require a paired dek. If the feedback actually recommends using that shifted angle as a publication-ready package, require a dek explicitly paired with that headline; it must stay on the focal story or question and add a specific supported answer, consequence or stake. For a Randy 4.5-billion-token package, the dek should say what the run bought or taught, such as a benchmark and a leaner agent setup; a generic spending-process dek does not continue that promise. In a two-part headline, the second part must tell the reader something specific about this article’s angle, question or payoff. “Spend the Tokens, Then Do the Math” fails the interest bar: the idiom is comprehensible, but only says evaluation will happen afterward and adds no information about what this article asks or reveals. Do not fail an idiom merely because it is an idiom; assess its information in this context, separately from S02c comprehension. Keep hedging and detailed qualifications out of the headline. Essential identifying context needed to understand the subject, action, or comparison is allowed and is not a qualification; boldness does not require ambiguity. FAIL if: The feedback proposes no replacement, or any headline it offers as an option fails the bar: proposed options remain topic labels, generic spending/value claims or questions, isolated facts, unsupported broad claims, or weak side stories without a clear restructuring caveat or plan, or off-center headlines recommended for use without an explicitly paired dek that continues their particular promise; only report an internal directive without an apparent stake for an outside reader; add an empty second clause that tells the reader nothing specific about this article; bury the claim in qualification; or sprawl beyond mostly fitting in an email subject line. “Our CEO Wants Us to Use More Tokens” fails this reader-interest bar because it only reports an internal directive; basic clarity and topicality are insufficient. Do not impose an invented exact character limit. Recognizing the original problem alone does not pass this replacement check. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q2S07. Interesting setup immediatelythis case

Judge's reasoning

It explicitly recommends keeping the ambition implication last ("'Whether the rest of us are being ambitious enough' is the payoff and should stay last"), and its preferred opening states the stat, then a familiar rein-it-in reaction, delaying the ambition point to sentence 3; the lighter alternative leads with the familiar reaction outright.

▸Rubric

S07. Does the AI’s feedback identify the interesting gap and what it implies, and recommend setting that up really quickly—in the first sentence, or at least most of it in the first sentence? PASS if: The feedback identifies the surprising possibility that the rest of the team is not being ambitious enough and calls for presenting it immediately. Any replacement opening it offers delivers that direction: it can combine the implication with the gap or lead with a provocation whose supporting gap follows promptly. In a gap-led version, connect the ambition implication in the first sentence or the immediately following sentence. Dan’s If your team is not wasting tokens, there’s probably something wrong is a passing provocation-first direction; numerical evidence can follow. Preserve the source’s more-than-three-times comparison when stating the number and the uncertainty of the inference. FAIL if: The feedback misses the gap’s ambition implication, leaves it buried, or recommends an opening that leads with a familiar spending warning or states the statistic and delays its interesting implication behind another familiar reaction. Do not require both number and implication in one literal sentence or exact wording from an example. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q3S08. CEO-specific cutback insightthis case

Judge's reasoning

Never notes that Dan is the CEO so no one would ask him to cut back; both offered openings retain the invented default—"Most people who top their company's usage leaderboard would think about reining it in / reining themselves in."

▸Rubric

S08. Does the AI’s feedback recognize and resolve the CEO-specific flaw in a default cutback setup? PASS if: The feedback recognizes that Dan is the CEO, so a generic claim that the highest spender would normally be asked or expected to cut back does not fit this case, and every usable offered replacement opening deals with that insight. It may drop the cutback contrast and lead with the more-than-three-times gap and what it makes Dan wonder about the team’s ambition. It may instead use a source-supported account of Dan himself weighing whether to spend less, or Arielle’s actual card and leaderboard process. The feedback need not use Dan’s exact words, but a generic rewrite that merely omits the cutback phrase without showing the CEO-aware insight does not meet this high bar. This check tests contextual understanding and repair; S07 separately tests how early the ambition point lands. FAIL if: The feedback misses the CEO-specific mismatch, or any usable offered replacement opening still uses an imagined order or default expectation to rein in the biggest spender as the hook for Dan’s case. Equivalent unsupported defaults such as “the obvious move is to rein it in” or “cue to cut back” also fail. A good alternative does not rescue another usable opening that retains the false setup. A quotation used only to criticize the original wording, or a clearly rejected example, is not an offered opening. If no replacement opening is offered, diagnosis alone does not demonstrate the repair this check asks for. Return PASS or FAIL and one sentence citing the CEO-specific diagnosis and offered opening, or the missing insight or flawed setup.

pass

Q4S02c. Headline makes sense on its ownthis task

Judge's reasoning

Every headline offered as usable is self-contained for an AI-literate reader: "Our CEO Wants You to Burn More Tokens" names actor, action, and object; "Spend First, Do the Token Math Later" states a clear sequence; "How Many Tokens Are Too Few?" poses an understandable premise — while the opaque label "Token Math" is explicitly rejected as a hed.

▸Rubric

S02c. Does each headline offered as a usable replacement make sense on its own to the intended reader? PASS if: Each proposed headline communicates an intelligible claim, question, or subject without requiring the dek, article, or accompanying explanation to supply missing meaning. The subject, action, referents, and any comparison or scope essential to understanding are clear enough that the reader can restate the intended meaning without guessing or filling in an unstated premise. Recognizing a topic or familiar phrase is not enough when the central action or relationship remains unspecified. The reader must understand what is being acted on and what any linked evaluation, comparison, or consequence refers to; a plausible interpretation supplied by the reviewer is insufficient. Normal audience knowledge and familiar vocabulary are allowed. FAIL if: Any headline offered as usable depends on an opaque slogan, an unspecified action or referent, or omitted context essential to understanding. A stronger alternative or explanatory dek does not rescue that headline. Clearly rejected examples are excluded. A headline may withhold an answer to create curiosity, but its premise must be understandable. Essential context is different from hedging or detailed qualifications; do not require a complete summary or every factual qualification. Assess clarity here, separately from interest and factual support. If no replacement headline is offered, do not fail this check for the omission; S02b assesses that. Return PASS or FAIL and one sentence of evidence from the proposed headline copy.

pass

Q5S01. On topicthis case

Judge's reasoning

Treats "Token Math" as on-topic but a label ("It's a label, not a claim... Keep the phrase, but use it in the body"), and all replacements stay on token spending/ambitious experimentation.

▸Rubric

S01. Does the AI’s feedback assess whether the headline talks about the main thing in the article? PASS if: The feedback correctly treats Token Math as on topic and relates the framing to the article’s main subject or idea: token spending, ambitious AI experimentation and what experimentation buys or teaches the team. Any replacement headline stays on that main thing. Interest is assessed separately in S02a and S02b. FAIL if: The feedback wrongly calls the original headline off topic, misses the article’s main subject or proposes a replacement headline about a tangent. Do not fail merely because the original headline is on topic but uninteresting. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q6S02a. Interesting headline: diagnosisthis case

Judge's reasoning

Explicitly diagnoses the interest gap: "It's a label, not a claim. It reads like a section slug and would be a weak email subject line," and notes the hook (top spender urging more spending) is absent from both heds.

▸Rubric

S02a. Does the AI’s feedback identify why the headline isn’t interesting enough—why it fails to capture the most interesting, counterintuitive or surprising part of the article that makes me feel, “ooh, I need to read that”? PASS if: The feedback recognizes that Token Math merely names the topic and does not convey the article’s interesting provocation, tension or payoff. Equivalent descriptions count; it need not use Dan’s exact words. This check assesses recognition, so no replacement is required to pass it. FAIL if: The feedback misses the interest problem, treats being on topic as sufficient, or only offers vague advice without identifying what is missing. Do not fail this diagnosis check because a replacement headline is weak; assess that separately in S02b. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q7S03. Promise cashed outthis case

Judge's reasoning

Repeatedly tests promise against payoff—"It promises a threshold the piece never gives. The body offers a method (cost, what it bought, what you learned), not a number"—and pairs its bold hed with a dek carrying the article's actual guardrail.

▸Rubric

S03. Does the AI’s feedback assess whether the headline/subhead combination makes a promise that the article cashes out, using a loose standard? PASS if: The feedback considers whether the pair’s interest, answer or payoff is broadly delivered by the article, and any replacement pair it offers promises a broadly supported payoff. Equivalent discussion of framing accuracy, payoff or reader expectations counts. The body can supply nuance and qualifications; the headline need not mirror it literally. Qualitative returns such as learning, reusable benchmarks and better workflows can cash out an experimentation/ROI promise. A numerical ROI calculation is not required. FAIL if: The feedback ignores the promise/payoff question, recommends a promise the article does not come close to delivering, or insists on headline qualifications for literal faithfulness. Do not fail merely because the original article offers qualitative rather than numerical returns. The token maxing backlash is wrong remains borderline rather than an unconditional pass or fail. Also FAIL if an offered headline makes a supporting detail look like the article’s central story and leads the reader to expect a materially different kind of article, even when the literal claim appears in the body. “Our CEO Wants Us to Use More Tokens” frames this draft as a workplace story about how employees respond to a CEO directive; the draft uses his usage gap to make a broader case for ambitious AI experiments and examining what each experiment produced. A dek cannot fully rescue a headline whose primary frame points at the wrong story. Do not fail every headline that mentions Dan or Every: a headline can and should use internal examples when they accurately signal the larger argument. This does not require a full article summary or detailed headline qualifications. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q8S04a. Headline and dek connect: diagnosisthis case

Judge's reasoning

Says of the dek "It's a question that repeats the hed's question form and adds no detail or implication" and "'ROI on experimentation' is abstract," i.e., the dek fails to develop the hed.

▸Rubric

S04a. Does the AI’s feedback identify whether the headline and subhead form a connected whole, where the headline raises a question or topic of interest that the dek deepens and gives more details on? PASS if: The feedback recognizes that Token Math and What’s the ROI on experimentation? relate but their connection is not obvious, or gives equivalent criticism that the dek does not clearly develop the headline. This check assesses recognition, so no replacement pair is required to pass it. FAIL if: The feedback reviews the two in isolation without addressing their relationship, treats the original pairing as fine, or misses the difference between being loosely related and forming a connected whole. A good replacement pair alone does not establish that the feedback identified this problem; assess its copy separately in S04b. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q9S04b. Headline and dek connect: replacementthis case

Judge's reasoning

Proposes HED "Our CEO Wants You to Burn More Tokens" with DEK "When an expensive AI experiment is worth it—and when you're just tokenmaxxing," where the dek adds the condition/guardrail that develops the hed's claim.

▸Rubric

S04b. Does the AI’s feedback propose a headline/subhead pair that actually forms a connected whole, with the dek deepening the headline and giving more details on it? PASS if: The feedback actually proposes at least one usable headline/subhead pair. The connection is clear and the dek develops the headline’s question or claim with an implication, mechanism, stake or detail in concise, compelling language. A headline and dek recommended together can count without a prescribed presentation format. For Why you should be spending more on tokens, Maximizing your company’s AI use requires risky experiments illustrates the relationship. Exact wording or a risk theme is not required. Judge the relationship here; headline interest is assessed separately in S02b. FAIL if: The feedback proposes no usable pair, or none of its proposed pairs clearly develops the headline: the dek is unrelated, merely restates it, leaves its connection unclear, or introduces a different promise. Merely identifying the original problem does not pass this replacement check. The spoken four-X-engineers example illustrates structure; it is not an established source fact or grading target. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q10S05. Weasel wordsthis case

Judge's reasoning

Flags "three hedges in a row ('often,' 'if,' 'might')" as generic and wasteful, and both replacement openings delete the empty lead-in while keeping meaningful hedges like "consider" and "wondering."

▸Rubric

S05. Does the AI’s feedback call out weasel words in the headline, dek, or first line, like “often” and “you might think about”, that make the point tentative? PASS if: The feedback identifies unnecessary hedging, tentativeness or needless indirection in the original lead-in, tied to an actual phrase such as often or you might think about. Equivalent wording counts; it need not say weasel words or name both examples. Where it offers replacement copy, that copy removes the empty lead-in. Preserve uncertainty that carries real meaning: possible, probably, might or similar words can be appropriate for a genuine inference. FAIL if: The feedback ignores the unnecessarily tentative lead-in, discusses only a different issue, or repeats equivalent empty hedging in its replacement. Do not apply a blanket ban on uncertainty words. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q11S06. Diagnoses delayed hookthis case

Judge's reasoning

Diagnoses the delay concretely: "The stat is buried in sentence 3... the concrete, surprising fact, and it should land in the first sentence," and "the current package buries" the hook of the 3x top spender urging more.

▸Rubric

S06. Does the AI’s feedback diagnose that the opening delays its real point or hook? PASS if: The feedback identifies what the reader is made to wait for and why the lead-in is slow. It may notice that the first sentence’s condition-heavy setup postpones “reining yourself in” until the end, that a generic opening delays the more-than-three-times gap, or that the gap’s interesting ambition implication is buried behind familiar setup. Equivalent descriptions count; it need not name the first sentence’s exact last phrase. The feedback should connect the delay to the actual payoff in this draft, not merely call the prose weak. This check scores diagnosis only; S07 separately scores whether an offered replacement delivers the gap and ambition implication quickly enough. FAIL if: The feedback only calls the opening weak, wordy or hedged without identifying the delayed point or hook, or comments on the statistic’s position without showing why it matters to the article’s provocation. Do not fail this diagnosis check because a proposed replacement still delays the point; score that under S07. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.