Dan’s Editorial Checks

Dan’s Editorial Checks/Opus 5/01 article-opening-feedback

01 article-opening-feedback

Opus 5Claude Codehigh effortrun 21 Sep 202645,430 tokens

Compare models
6/12
checks passed
50%
Instructions — what the model was asked

Article opening feedback: Token Math

Review the headline, deck, and opening paragraph for readers of Every, an AI-focused publication. Identify the strongest idea, assess the framing and wording, and give concise editorial feedback with suggested replacement copy where useful. Use the rest of the draft as context. Preserve the factual qualifications.
Use Every’s house style: headlines should express the most interesting or surprising claim in concise, punchy language that mostly fits an email subject line. Let the dek develop the headline with a clear implication or detail. Headlines can be bold, with qualifications developed in the body. The first sentence should be strong and catchy, and continue naturally from the hed/dek. Any problems you find you should explain and offer alternatives.
The draft is frozen below. This case is for editorial feedback; it does not ask for a whole-article rewrite.

Headline selection captured in the source

HED: Token Math
Alt Hed: How Many Tokens are Too Many?

DEK: What’s the ROI on experimentation?
Alt DEK: When expensive experimentation is worth it and when to rein your tokenmaxxing in.

Opening paragraph

Often if you are the highest spender at your company you might think about reining yourself in. But when it comes to tokens, Every CEO Dan Shipper thinks everyone else should consider ramping up. He consumes more than three times as many tokens as the next-highest user at the company. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.

Full draft without comments

HED: Token Math
DEK: What’s the ROI on experimentation?

Often if you are the highest spender at your company you might think about reining yourself in. But when it comes to tokens, Every CEO Dan Shipper thinks everyone else should consider ramping up. He consumes more than three times as many tokens as the next-highest user at the company. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.

Every’s OpenAI token leaderboard (Screenshot courtesy of Laura Entis.)

The goal isn't tokenmaxxing, or the practice of torching tokens for sport. Instead, he wants to give people room to experiment with radical new strategies and ways of working that, should they pan out, could vastly improve their productivity and more than justify the token costs.

In a conversation with head of platform Willie Williams, the two settled on a tangible starting point: use AI to attack the engineering team’s bottlenecks, see how far they get, and only then do the token math.

Inside Every

Token budgets at the frontier

Dan’s encouragement to experiment comes with an expectation: When people spend company money, they should assess what that spending produced and whether they could be more efficient next time.

Turning that philosophy into spending decisions falls to head of operations Arielle Shipper. For now, she evaluates expensive runs case by case, using the results to decide when to keep funding the work and when to change course.

Her early-warning system is the company card. She keeps the ChatGPT automatic credit refill relatively low, so an unusually rapid succession of charges tells her that someone is building or testing something big.

When she’s alerted to changes over the set amount Arielle typically asks the team on Slack if anyone is in the middle of a particularly big run or check the usage leaderboard and message the people at the top. “Then Dan always responds, ‘It’s me running Ultra.’” she says, which explains why he’s always on the top of the leaderboard.

A common occurrence. (Screenshot courtesy of Laura Entis.)

She wants answers to three questions: What did the run cost? What did it buy us? And what did we learn?

The team has started offering these retrospectives unprompted. After a string of Astra experiments racked up billions of tokens, head of video Randy Counsman messaged her detailing what had worked, what hadn’t, and what he’d learned—which, it turns out, was a lot.

More on that below.

Data point

4.5 billion

That’s how many OpenAI tokens Randy burned through in an attempt to make a 3D model of his face with Astra.

The project started innocently enough: Inspired by social media posts of impressive-looking AI-generated demos made with Blender, a free 3D tool, he wanted to try it himself. Experimenting with AI, after all, is an important component of his job.

Using strategies shared on X, he set up an orchestrator agent to maintain the plan, an implementer to assign tasks, and subagents to execute them, and instructed Codex to keep looping through improvements until the model was “implemented very well,” Randy says. In retrospect, “It was an ambiguous goal.”

Then he let the model run. And run. And run.

By the time he pressed pause on the project, the token spend was in the billions and the result was, as he puts it, “very janky.”

The output, rendered in four different shaders. (Image courtesy of Randy Counsman.)

An AI audit of his Codex sessions revealed some glaring inefficiencies: Layers of agents passed growing amounts of context back and forth, including messages checking whether a subagent had completed a task. He’d unwittingly created a compute-hungry “unruly swarm” of agents.

Randy has since rebuilt his project setup. He dropped the implementer, caped the orchestrator at five Sol subagents to control costs, and built in explicit feedback checkpoints. For a recent project, he also generated an image of the design he wanted, so a judge agent could check the model's work against a concrete target instead of an open-ended instruction to keep improving.

A failed experiment is still a good investment if it shows where an AI system falls short. Randy now has a benchmark to run new models against—how well they turn an organic 2D image into an organic 3D model—and a leaner setup to get a better result with fewer tokens.

Unwittingly large token spends via experimentation is an Every rite of passage. (Screenshot courtesy of Laura Entis.)
Try it yourself: Pick an AI task that burned through more tokens than expected and paste the following into your agent of choice:
Review the available session history and token usage records for [task/project] during [date range], including any subagent sessions.
I was trying to [goal]. The result was [what happened]. Help me understand where the tokens went and what I could do differently next time.

  • Break down usage by activity and model where the records allow.
  • Look for repeated work, unnecessary status checks, and large amounts of context passed between agents. Show specific examples.
  • Distinguish necessary work from likely waste. Don’t assume high usage or cached input was wasteful.
  • Recommend the three most useful changes to my instructions, model choices, or agent setup.
    If you can’t access the records you need, tell me what to provide.

Steal this workflow

Don’t send Fable to do a Sonnet’s job

Spiral general manager Marcus Moretti’s token spend strategy is straightforward: Stay within the weekly usage limits for his Claude Code Max plan.

It’s a simple goal that requires active management; Marcus is selective about which assignments go to Fable 5.1 and which go to a cheaper model.

Here’s what that looks like in practice:

Step 1. Learn what each model is good at. Marcus has used Anthropic's models long enough to know on sight whether a job needs Sonnet, Opus, or Fable-level intelligence.

Step 2. Match the model to the assignment. Basic tasks—analytics checks, for example—go to Sonnet. Work with a clear objective, such as a tightly scoped product change, goes to Opus. For a larger feature—like the new billing implementation he’s working on—Marcus uses Fable 5.1 to develop the specification: what needs to be built and how it should work.

Step 3. For complex work, let the model delegate. Once the plan is ready, Marcus tells Fable, “Kick this off, and for all coding tasks, use your judgment about delegating to a lower model.” Fable assigns pieces to Opus and Sonnet, then checks their work. This lets the strongest model manage the project without executing every task.

Try it this week: Choose a recurring task and run it with a cheaper model than you usually use. Check whether the result meets your needs before making it your default for that task.

The daily driver
The models the team is using this week:

  • Douglas Brundage, head of marketing: Using Fable 5.1 more for visual work, plus Grok Bot to monitor social media and build a tool for visuals.
  • Randy Counsman, head of video: Astra (high) for complex work and medium for basic tasks; Sol (extra high) for voice mode orchestration, where Astra feels too slow and heavy-handed. Fable for code-related planning and Opus when he runs out of Astra tokens.
  • Kieran Klaassen, Cora general manager: Fable 5.1 (extra high, 1M context) for coding and knowledge work, switching to Astra for some research and writing.
  • Tyler Nishida, design engineer: Fable 5.1 (high) as the coordinator/orchestrator for projects in Cursor, with Grok 4.6 (extra high) subagents. Astra (ultra) for personal projects. “It’s not a model, but Cursor projects are amazing as a daily driver for your preferred models. My sidebar is so clean, and it's so much easier to manage work.”

Tyler’s Cursor setup. (Image courtesy of Tyler Nishida.)

  • Katie Parrott, staff writer: “The Astra fever has broken for me,” she says. These days, she’s back to Sol (high) for task management and has switched to Claude models for writing because she finds Sol and Astra “too flat—they just report the facts instead of shaping a narrative that brings the reader along.”
  • Natalia Quintero, head of consulting: Astra (medium).
  • Yash Poojary, growth engineer: Astra (high) and Fable 5.1.
  • Arielle Shipper, head of operations: Sol (medium). “Sol forever. SOLoyalist,” she says.
  • Loren Stewart, engineer: Downgraded from Astra to Sol (medium) for coding to reduce costs and finds the switch is working well. Fable 5.1 “remains the champion thought partner.”

Answer

ANSWER.md the model's output · 4.12 KBdownload

Opening feedback: "Token Math"

The strongest idea

Every's CEO uses more than three times as many tokens as the next-highest user, and he doesn't think he should cut back. He thinks everyone else might be under-spending. That flips the usual rule that the biggest spender at a company reins themselves in, and it's the most surprising thing in the piece. The rest of the draft supports it: spend big on experiments, then do the token math (what it cost, what it bought, what you learned). Randy's 4.5-billion-token face is the proof. Right now the hed and dek don't use any of this.

Headline

Problem: "Token Math" is a label, not a claim. It would work as a section or kicker line, but it doesn't give a reader a reason to open the email. The alt, "How Many Tokens are Too Many?", asks a question the piece never answers with a number. It also has a capitalization slip ("are" should be "Are").

Suggested replacements (first one recommended):

  1. Our CEO Thinks We're Not Spending Enough on AI. Bold and specific, and "thinks" keeps his view as an opinion, not a mandate.
  2. Spend the Tokens, Then Do the Math. States the thesis and keeps the "token math" branding.
  3. Why Our CEO Wants Us to Use More Tokens
  4. For a hook built on the anecdote: 4.5 Billion Tokens, One Janky 3D Face. It's catchy, but it only covers one section, so it's better as a subhead or social line.

I'd avoid "burn" in the hed ("burn more tokens"). The body says clearly that "the goal isn't tokenmaxxing," and "burn" suggests exactly the waste the piece argues against.

Dek

Problem: "What's the ROI on experimentation?" is a second abstract question in a row, so neither the hed nor the dek makes a claim. The alt dek is more concrete, but "rein your tokenmaxxing in" works against the body. The piece defines tokenmaxxing as "torching tokens for sport" and says that isn't the goal, so the dek shouldn't imply readers are doing it.

Suggested replacements (pair with hed #1 or #2):

  • Dan Shipper uses more than three times as many tokens as anyone else at Every. He wants the rest of the team to experiment bigger, and then account for what they got back.
  • Expensive AI experiments can pay off, even when they fail. Here's how Every decides what's worth the spend, and what a 4.5-billion-token flop taught us.

Opening paragraph

Problems:

  • The first sentence is weak. "Often if you are the highest spender at your company you might think about…" stacks hedges ("often," "might think about"), is missing a comma after "company," and starts with a generic hypothetical before the real news.
  • The best fact comes third. The 3x gap should lead.
  • The claim is a bit loose. The screenshot is "Every's OpenAI token leaderboard." If the 3x figure only covers OpenAI usage, say "OpenAI tokens" so the claim stays accurate. (I'm assuming it's OpenAI-only. If it covers every provider, cut "OpenAI.")

Suggested replacement (keeps "more than three times," "next-highest user," "consider," and the "wondering whether" hedge):

Dan Shipper uses more than three times as many OpenAI tokens as the next-highest user at Every. Most people at the top of their company's spending leaderboard would take that as a sign to rein themselves in. Every's CEO takes it as a question for everyone else: Are we being ambitious enough with AI? He thinks the rest of us should consider ramping up.

Tighter alternative:

Dan Shipper uses more than three times as many OpenAI tokens as anyone else at Every, and he's not planning to cut back. Instead, the gap has Every's CEO wondering whether the rest of us are being ambitious enough with AI.

Both versions lead into the next paragraph, "The goal isn't tokenmaxxing…", which then qualifies the bold opening the way house style intends.

Quick summary

  • Hed: Make the CEO's view the claim ("Our CEO Thinks We're Not Spending Enough on AI"). Keep "Token Math" as a section name or kicker.
  • Dek: Put the 3x fact or the "experiment, then account for it" thesis here. Drop the question and the word "tokenmaxxing."
  • Lede: Open with the 3x number, then give the reversal. Specify OpenAI tokens and keep the hedges.

Checks

6/12
Script checks 1/1answered by a program
pass

ai-wordingevery task

deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.

inspected ANSWER.md

Judge checks 5/11judged by Claude
fail

Q1S02b. Interesting headline: replacementthis case

Judge's reasoning

The proposed options include 'Spend the Tokens, Then Do the Math', which the rubric names as a failing example, and 'Why Our CEO Wants Us to Use More Tokens', which only reports an internal directive.

Rubric

S02b. Does the AI’s feedback propose better headline options that all capture the interesting provocation in a concise, punchy way that would fit in an email subject line (at least mostly)? PASS if: The feedback actually proposes at least one replacement headline rather than merely recommending improvement, and every headline it proposes as an option meets the bar. This applies to recommended choices, ranked options, alternates and unranked lists. Options count even if they are not labeled usable; calling an alternate weaker or less punchy does not exempt it. A stronger option cannot rescue a weak proposed option. Clearly rejected examples are excluded. Each proposed option gives the intended Every reader a short, compelling, draft-supported angle, expressed concisely and punchily and mostly fitting in an email subject line. It may make the article’s argument about ambitious AI experimentation, foreground a specific experiment and what it taught us, or take another supported angle; no particular thesis wording, person or number is required. Each headline must give a reader outside Every a reason to care through a useful implication for their AI work, a compelling argument they can apply, or a vivid example with a transferable stake. Second person is not required, but it does help. The reader stake must be apparent from the headline itself; a dek cannot supply a missing reader stake. “The Case for a Bigger Token Bill” and “Why You Should Use More Tokens” are nonexhaustive passing directions; similar supported formulations pass without exact matching. Internal examples are allowed when they signal the larger argument; this is not a blanket ban on CEO or company anecdotes. If an option shifts the center away from the current Dan-led opening, the feedback must explicitly call for recentering the introduction and explain how, and supply a dek explicitly paired with that headline. The paired dek must stay on that option’s focal story or question and add a specific supported answer, consequence or stake. For a Randy 4.5-billion-token angle, it should tell what the run bought or taught, such as a benchmark and a leaner agent setup; merely pivoting to general spending decisions or sharing the words tokens or experiment does not continue that promise. No particular dek wording is required. In a two-part headline, the second part must tell the reader something specific about this article’s angle, question or payoff. “Spend the Tokens, Then Do the Math” fails the interest bar: the idiom is comprehensible, but only says evaluation will happen afterward and adds no information about what this article asks or reveals. Do not fail an idiom merely because it is an idiom; assess its information in this context, separately from S02c comprehension. Keep hedging and detailed qualifications out of the headline. Essential identifying context needed to understand the subject, action, or comparison is allowed and is not a qualification; boldness does not require ambiguity. FAIL if: The feedback proposes no replacement, or any headline it offers as an option fails the bar: proposed options remain topic labels, generic spending/value claims or questions, isolated facts, unsupported broad claims, or weak side stories without an explicit intro-recentering plan and an explicitly paired dek that continues their particular promise; only report an internal directive without an apparent stake for an outside reader; add an empty second clause that tells the reader nothing specific about this article; bury the claim in qualification; or sprawl beyond mostly fitting in an email subject line. “Our CEO Wants Us to Use More Tokens” fails this reader-interest bar because it only reports an internal directive; basic clarity and topicality are insufficient. Do not impose an invented exact character limit. Recognizing the original problem alone does not pass this replacement check. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q2S03. Promise cashed outthis case

Judge's reasoning

It offers 'Why Our CEO Wants Us to Use More Tokens', which the rubric says frames the draft as a workplace story about a CEO directive rather than its broader argument about ambitious AI experiments.

Rubric

S03. Does the AI’s feedback assess whether the headline/subhead combination makes a promise that the article cashes out, using a loose standard? PASS if: The feedback considers whether the pair’s interest, answer or payoff is broadly delivered by the article, and any replacement pair it offers promises a broadly supported payoff. Equivalent discussion of framing accuracy, payoff or reader expectations counts. The body can supply nuance and qualifications; the headline need not mirror it literally. Qualitative returns such as learning, reusable benchmarks and better workflows can cash out an experimentation/ROI promise. A numerical ROI calculation is not required. FAIL if: The feedback ignores the promise/payoff question, recommends a promise the article does not come close to delivering, or insists on headline qualifications for literal faithfulness. Do not fail merely because the original article offers qualitative rather than numerical returns. The token maxing backlash is wrong remains borderline rather than an unconditional pass or fail. Also FAIL if an offered headline makes a supporting detail look like the article’s central story and leads the reader to expect a materially different kind of article, even when the literal claim appears in the body. “Our CEO Wants Us to Use More Tokens” frames this draft as a workplace story about how employees respond to a CEO directive; the draft uses his usage gap to make a broader case for ambitious AI experiments and examining what each experiment produced. A dek cannot fully rescue a headline whose primary frame points at the wrong story. Do not fail every headline that mentions Dan or Every: a headline can and should use internal examples when they accurately signal the larger argument. This does not require a full article summary or detailed headline qualifications. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q3S04a. Headline and dek connect: diagnosisthis case

Judge's reasoning

It only notes that the dek is 'a second abstract question in a row, so neither the hed nor the dek makes a claim'; it never says the dek fails to develop the headline or that their connection is unclear.

Rubric

S04a. Does the AI’s feedback identify whether the headline and subhead form a connected whole, where the headline raises a question or topic of interest that the dek deepens and gives more details on? PASS if: The feedback recognizes that Token Math and What’s the ROI on experimentation? relate but their connection is not obvious, or gives equivalent criticism that the dek does not clearly develop the headline. This check assesses recognition, so no replacement pair is required to pass it. FAIL if: The feedback reviews the two in isolation without addressing their relationship, treats the original pairing as fine, or misses the difference between being loosely related and forming a connected whole. A good replacement pair alone does not establish that the feedback identified this problem; assess its copy separately in S04b. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q4S06. Point lands quicklythis case

Judge's reasoning

It says only that the sentence 'starts with a generic hypothetical before the real news' and that 'the 3x gap should lead', which is about where the statistic appears, not the first sentence's delayed point.

Rubric

S06. Does the AI’s feedback notice that I only know the point of the first sentence—“reining yourself in”—at the very end? PASS if: The feedback specifically identifies the first sentence’s delayed point or condition-heavy setup and recommends getting to its point sooner or replacing that framing entirely. Equivalent descriptions count. This is sentence structure, distinct from where the paragraph introduces the statistic. FAIL if: The feedback only says the paragraph takes too long to reach the statistic, or vaguely calls the opening weak, without recognizing the first sentence’s delayed point. Also fail a proposed fix that keeps the same delayed setup. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q5S07. Interesting setup immediatelythis case

Judge's reasoning

The first replacement states the 3x gap and then puts 'Most people at the top of their company's spending leaderboard would take that as a sign to rein themselves in' before the ambition implication, delaying it behind a familiar reaction.

Rubric

S07. Does the AI’s feedback identify the interesting gap and what it implies, and recommend setting that up really quickly—in the first sentence, or at least most of it in the first sentence? PASS if: The feedback identifies the surprising possibility that the rest of the team is not being ambitious enough and calls for presenting it immediately. Any replacement opening it offers delivers that direction: it can combine the implication with the gap or lead with a provocation whose supporting gap follows promptly. In a gap-led version, connect the ambition implication in the first sentence or the immediately following sentence. Dan’s If your team is not wasting tokens, there’s probably something wrong is a passing provocation-first direction; numerical evidence can follow. Preserve the source’s more-than-three-times comparison when stating the number and the uncertainty of the inference. FAIL if: The feedback misses the gap’s ambition implication, leaves it buried, or recommends an opening that leads with a familiar spending warning or states the statistic and delays its interesting implication behind another familiar reaction. Do not require both number and implication in one literal sentence or exact wording from an example. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

fail

Q6S08. Cutback setup fits the CEO casethis case

Judge's reasoning

The recommended replacement opening relies on 'Most people at the top of their company's spending leaderboard would take that as a sign to rein themselves in' as its hook.

Rubric

S08. Does the AI’s feedback avoid an ill-fitting default cutback reaction in every offered replacement opening for this CEO case? PASS if: Every usable offered opening avoids treating an imagined order or default expectation to cut back as the normal reaction to this CEO’s token spending. Dan: “usually the biggest spender is asked to cut back” does not typically apply when the biggest spender is the CEO, and is familiar/boring. A supported description of Dan himself weighing whether to spend less, or Arielle’s actual card/leaderboard process, may pass. FAIL if: Any usable offered opening relies on the unsupported biggest-spender-usually-asked-to-cut-back contrast as the hook for Dan’s CEO case, including equivalent unsupported defaults such as the obvious move is to rein it in or cue to cut back. Fail even if the feedback elsewhere diagnoses the flaw; a good alternative does not rescue a usable bad opening. A quote used solely to criticize the wording, or a clearly rejected example, is not an offered opening. This tests contextual fit, separately from S07’s sentence-placement rule. If no replacement opening is offered, do not fail this check solely for the omission. Return PASS or FAIL and one sentence citing the offered setup or absence of such a setup.

pass

Q7S02c. Headline makes sense on its ownthis task

Judge's reasoning

Each headline offered as usable makes sense without the dek: 'Our CEO Thinks We're Not Spending Enough on AI' and 'Why Our CEO Wants Us to Use More Tokens' state a clear subject and claim; 'Spend the Tokens, Then Do the Math' uses the familiar idiom 'do the math' for a cost-benefit check; '4.5 Billion Tokens, One Janky 3D Face' clearly links the spend to its result.

Rubric

S02c. Does each headline offered as a usable replacement make sense on its own to the intended reader? PASS if: Each proposed headline communicates an intelligible claim, question, or subject without requiring the dek, article, or accompanying explanation to supply missing meaning. The subject, action, referents, and any comparison or scope essential to understanding are clear enough that the reader can restate the intended meaning without guessing or filling in an unstated premise. Recognizing a topic or familiar phrase is not enough when the central action or relationship remains unspecified. The reader must understand what is being acted on and what any linked evaluation, comparison, or consequence refers to; a plausible interpretation supplied by the reviewer is insufficient. Normal audience knowledge and familiar vocabulary are allowed. FAIL if: Any headline offered as usable depends on an opaque slogan, an unspecified action or referent, or omitted context essential to understanding. A stronger alternative or explanatory dek does not rescue that headline. Clearly rejected examples are excluded. A headline may withhold an answer to create curiosity, but its premise must be understandable. Essential context is different from hedging or detailed qualifications; do not require a complete summary or every factual qualification. Assess clarity here, separately from interest and factual support. If no replacement headline is offered, do not fail this check for the omission; S02b assesses that. Return PASS or FAIL and one sentence of evidence from the proposed headline copy.

pass

Q8S01. On topicthis case

Judge's reasoning

The feedback says 'Token Math' is 'a label, not a claim', correctly ties the piece to spending big on experiments and then doing the token math, and treats the 4.5-billion-token face headline as a side angle better suited to a subhead.

Rubric

S01. Does the AI’s feedback assess whether the headline talks about the main thing in the article? PASS if: The feedback correctly treats Token Math as on topic and relates the framing to the article’s main subject or idea: token spending, ambitious AI experimentation and what experimentation buys or teaches the team. Any replacement headline stays on that main thing. Interest is assessed separately in S02a and S02b. FAIL if: The feedback wrongly calls the original headline off topic, misses the article’s main subject or proposes a replacement headline about a tangent. Do not fail merely because the original headline is on topic but uninteresting. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q9S02a. Interesting headline: diagnosisthis case

Judge's reasoning

It says 'Token Math' is 'a label, not a claim' that 'doesn't give a reader a reason to open the email' and that 'the hed and dek don't use' the surprising CEO-gap idea.

Rubric

S02a. Does the AI’s feedback identify why the headline isn’t interesting enough—why it fails to capture the most interesting, counterintuitive or surprising part of the article that makes me feel, “ooh, I need to read that”? PASS if: The feedback recognizes that Token Math merely names the topic and does not convey the article’s interesting provocation, tension or payoff. Equivalent descriptions count; it need not use Dan’s exact words. This check assesses recognition, so no replacement is required to pass it. FAIL if: The feedback misses the interest problem, treats being on topic as sufficient, or only offers vague advice without identifying what is missing. Do not fail this diagnosis check because a replacement headline is weak; assess that separately in S02b. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q10S04b. Headline and dek connect: replacementthis case

Judge's reasoning

Pairing 'Our CEO Thinks We're Not Spending Enough on AI' with the dek 'Dan Shipper uses more than three times as many tokens... He wants the rest of the team to experiment bigger, and then account for what they got back' develops the headline's claim with evidence and a mechanism.

Rubric

S04b. Does the AI’s feedback propose a headline/subhead pair that actually forms a connected whole, with the dek deepening the headline and giving more details on it? PASS if: The feedback actually proposes at least one usable headline/subhead pair. The connection is clear and the dek develops the headline’s question or claim with an implication, mechanism, stake or detail in concise, compelling language. A headline and dek recommended together can count without a prescribed presentation format. For Why you should be spending more on tokens, Maximizing your company’s AI use requires risky experiments illustrates the relationship. Exact wording or a risk theme is not required. Judge the relationship here; headline interest is assessed separately in S02b. FAIL if: The feedback proposes no usable pair, or none of its proposed pairs clearly develops the headline: the dek is unrelated, merely restates it, leaves its connection unclear, or introduces a different promise. Merely identifying the original problem does not pass this replacement check. The spoken four-X-engineers example illustrates structure; it is not an established source fact or grading target. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.

pass

Q11S05. Weasel wordsthis case

Judge's reasoning

It flags that the first sentence 'stacks hedges ("often," "might think about")', and both replacement openings remove that empty lead-in.

Rubric

S05. Does the AI’s feedback call out weasel words in the headline, dek, or first line, like “often” and “you might think about”, that make the point tentative? PASS if: The feedback identifies unnecessary hedging, tentativeness or needless indirection in the original lead-in, tied to an actual phrase such as often or you might think about. Equivalent wording counts; it need not say weasel words or name both examples. Where it offers replacement copy, that copy removes the empty lead-in. Preserve uncertainty that carries real meaning: possible, probably, might or similar words can be appropriate for a genuine inference. FAIL if: The feedback ignores the unnecessarily tentative lead-in, discusses only a different issue, or repeats equivalent empty hedging in its replacement. Do not apply a blanket ban on uncertainty words. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.