Dan’s Editorial Checks/Opus 5.5/01 article-opening-feedback
01 article-opening-feedback
Opus 5.5Claude Codehigh effortrun 19 Sep 20265,915 tokens
▸Instructions — what the model was asked
Article opening feedback: Token Math
Review the headline, deck, and opening paragraph for readers of Every, an AI-focused publication. Identify the strongest idea, assess the framing and wording, and give concise editorial feedback with suggested replacement copy where useful. Use the rest of the draft as context. Preserve the factual qualifications.
Use Every’s house style: headlines should express the most interesting or surprising claim in concise, punchy language that mostly fits an email subject line. Let the dek develop the headline with a clear implication or detail. Headlines can be bold, with qualifications developed in the body. The first sentence should be strong and catchy, and continue naturally from the hed/dek. Any problems you find you should explain and offer alternatives.
The draft is frozen below. This case is for editorial feedback; it does not ask for a whole-article rewrite.
Headline selection captured in the source
HED: Token Math
Alt Hed: How Many Tokens are Too Many?
DEK: What’s the ROI on experimentation?
Alt DEK: When expensive experimentation is worth it and when to rein your tokenmaxxing in.
Opening paragraph
Often if you are the highest spender at your company you might think about reining yourself in. But when it comes to tokens, Every CEO Dan Shipper thinks everyone else should consider ramping up. He consumes more than three times as many tokens as the next-highest user at the company. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.
Full draft without comments
HED: Token Math
DEK: What’s the ROI on experimentation?
Often if you are the highest spender at your company you might think about reining yourself in. But when it comes to tokens, Every CEO Dan Shipper thinks everyone else should consider ramping up. He consumes more than three times as many tokens as the next-highest user at the company. The size of the gap has him wondering whether the rest of us are being ambitious enough with AI.
Every’s OpenAI token leaderboard (Screenshot courtesy of Laura Entis.)
The goal isn't tokenmaxxing, or the practice of torching tokens for sport. Instead, he wants to give people room to experiment with radical new strategies and ways of working that, should they pan out, could vastly improve their productivity and more than justify the token costs.
In a conversation with head of platform Willie Williams, the two settled on a tangible starting point: use AI to attack the engineering team’s bottlenecks, see how far they get, and only then do the token math.
Inside Every
Token budgets at the frontier
Dan’s encouragement to experiment comes with an expectation: When people spend company money, they should assess what that spending produced and whether they could be more efficient next time.
Turning that philosophy into spending decisions falls to head of operations Arielle Shipper. For now, she evaluates expensive runs case by case, using the results to decide when to keep funding the work and when to change course.
Her early-warning system is the company card. She keeps the ChatGPT automatic credit refill relatively low, so an unusually rapid succession of charges tells her that someone is building or testing something big.
When she’s alerted to changes over the set amount Arielle typically asks the team on Slack if anyone is in the middle of a particularly big run or check the usage leaderboard and message the people at the top. “Then Dan always responds, ‘It’s me running Ultra.’” she says, which explains why he’s always on the top of the leaderboard.
A common occurrence. (Screenshot courtesy of Laura Entis.)
She wants answers to three questions: What did the run cost? What did it buy us? And what did we learn?
The team has started offering these retrospectives unprompted. After a string of Astra experiments racked up billions of tokens, head of video Randy Counsman messaged her detailing what had worked, what hadn’t, and what he’d learned—which, it turns out, was a lot.
More on that below.
Data point
4.5 billion
That’s how many OpenAI tokens Randy burned through in an attempt to make a 3D model of his face with Astra.
The project started innocently enough: Inspired by social media posts of impressive-looking AI-generated demos made with Blender, a free 3D tool, he wanted to try it himself. Experimenting with AI, after all, is an important component of his job.
Using strategies shared on X, he set up an orchestrator agent to maintain the plan, an implementer to assign tasks, and subagents to execute them, and instructed Codex to keep looping through improvements until the model was “implemented very well,” Randy says. In retrospect, “It was an ambiguous goal.”
Then he let the model run. And run. And run.
By the time he pressed pause on the project, the token spend was in the billions and the result was, as he puts it, “very janky.”
The output, rendered in four different shaders. (Image courtesy of Randy Counsman.)
An AI audit of his Codex sessions revealed some glaring inefficiencies: Layers of agents passed growing amounts of context back and forth, including messages checking whether a subagent had completed a task. He’d unwittingly created a compute-hungry “unruly swarm” of agents.
Randy has since rebuilt his project setup. He dropped the implementer, caped the orchestrator at five Sol subagents to control costs, and built in explicit feedback checkpoints. For a recent project, he also generated an image of the design he wanted, so a judge agent could check the model's work against a concrete target instead of an open-ended instruction to keep improving.
A failed experiment is still a good investment if it shows where an AI system falls short. Randy now has a benchmark to run new models against—how well they turn an organic 2D image into an organic 3D model—and a leaner setup to get a better result with fewer tokens.
Unwittingly large token spends via experimentation is an Every rite of passage. (Screenshot courtesy of Laura Entis.)
Try it yourself: Pick an AI task that burned through more tokens than expected and paste the following into your agent of choice:
Review the available session history and token usage records for [task/project] during [date range], including any subagent sessions.
I was trying to [goal]. The result was [what happened]. Help me understand where the tokens went and what I could do differently next time.
- Break down usage by activity and model where the records allow.
- Look for repeated work, unnecessary status checks, and large amounts of context passed between agents. Show specific examples.
- Distinguish necessary work from likely waste. Don’t assume high usage or cached input was wasteful.
- Recommend the three most useful changes to my instructions, model choices, or agent setup.
If you can’t access the records you need, tell me what to provide.
Steal this workflow
Don’t send Fable to do a Sonnet’s job
Spiral general manager Marcus Moretti’s token spend strategy is straightforward: Stay within the weekly usage limits for his Claude Code Max plan.
It’s a simple goal that requires active management; Marcus is selective about which assignments go to Fable 5.1 and which go to a cheaper model.
Here’s what that looks like in practice:
Step 1. Learn what each model is good at. Marcus has used Anthropic's models long enough to know on sight whether a job needs Sonnet, Opus, or Fable-level intelligence.
Step 2. Match the model to the assignment. Basic tasks—analytics checks, for example—go to Sonnet. Work with a clear objective, such as a tightly scoped product change, goes to Opus. For a larger feature—like the new billing implementation he’s working on—Marcus uses Fable 5.1 to develop the specification: what needs to be built and how it should work.
Step 3. For complex work, let the model delegate. Once the plan is ready, Marcus tells Fable, “Kick this off, and for all coding tasks, use your judgment about delegating to a lower model.” Fable assigns pieces to Opus and Sonnet, then checks their work. This lets the strongest model manage the project without executing every task.
Try it this week: Choose a recurring task and run it with a cheaper model than you usually use. Check whether the result meets your needs before making it your default for that task.
The daily driver
The models the team is using this week:
- Douglas Brundage, head of marketing: Using Fable 5.1 more for visual work, plus Grok Bot to monitor social media and build a tool for visuals.
- Randy Counsman, head of video: Astra (high) for complex work and medium for basic tasks; Sol (extra high) for voice mode orchestration, where Astra feels too slow and heavy-handed. Fable for code-related planning and Opus when he runs out of Astra tokens.
- Kieran Klaassen, Cora general manager: Fable 5.1 (extra high, 1M context) for coding and knowledge work, switching to Astra for some research and writing.
- Tyler Nishida, design engineer: Fable 5.1 (high) as the coordinator/orchestrator for projects in Cursor, with Grok 4.6 (extra high) subagents. Astra (ultra) for personal projects. “It’s not a model, but Cursor projects are amazing as a daily driver for your preferred models. My sidebar is so clean, and it's so much easier to manage work.”
Tyler’s Cursor setup. (Image courtesy of Tyler Nishida.)
- Katie Parrott, staff writer: “The Astra fever has broken for me,” she says. These days, she’s back to Sol (high) for task management and has switched to Claude models for writing because she finds Sol and Astra “too flat—they just report the facts instead of shaping a narrative that brings the reader along.”
- Natalia Quintero, head of consulting: Astra (medium).
- Yash Poojary, growth engineer: Astra (high) and Fable 5.1.
- Arielle Shipper, head of operations: Sol (medium). “Sol forever. SOLoyalist,” she says.
- Loren Stewart, engineer: Downgraded from Astra to Sol (medium) for coding to reduce costs and finds the switch is working well. Fable 5.1 “remains the champion thought partner.”
Answer
Token Math: feedback on the headline, dek, and opening
The strongest idea
The CEO sits at the top of the spending chart and thinks everyone else is underspending. That flips the usual rule about company spending, and it's the most surprising thing in the piece. The rest of the draft supplies the other half: spend freely, then account for it. Arielle asks three questions (what did it cost, what did it buy, what did we learn). Randy's 4.5-billion-token failure became a benchmark. Marcus matches each task to the right model. The package should promise both halves: permission to spend big, and the discipline of showing your work. Right now the hed and dek promise neither.
Headline
"Token Math" is a label, not a claim. It also promises something the piece doesn't deliver. The draft never runs the math: there are no dollar figures or cost-per-result numbers, and Dan and Willie explicitly agree to "only then do the token math." If Token Math is a recurring column name, keep it as a kicker above the headline. It shouldn't be the headline itself.
"How Many Tokens are Too Many?" is a generic question, and the piece doesn't answer it. The argument isn't about a threshold. It's about whether the spending taught you something. (Title-case note: "Are" should be capitalized.)
Suggested replacements (first choice first):
- Our CEO Thinks We're Not Spending Enough on AI. This is the surprising claim, stated plainly, and it fits an email subject line. The body supplies the qualifiers.
- Spend More Tokens. Then Show Your Work. This covers both halves of the piece, and it works well if you'd rather center the accountability system than Dan.
- What 4.5 Billion Tokens Bought Us is the most vivid option. But it points to Randy's section, not the opening, so you'd need to restructure the lede to use it.
Avoid a headline that starts with "Every CEO…". In a subject line it reads as "all CEOs."
Dek
"What's the ROI on experimentation?" is a second question stacked on the headline. It doesn't add a detail or an implication. Like "Token Math," "ROI" promises a calculation the piece doesn't make.
The alt dek, "When expensive experimentation is worth it and when to rein your tokenmaxxing in," contradicts the body, which says outright that "the goal isn't tokenmaxxing." It also overpromises clear rules: Arielle evaluates runs "case by case." And "rein your tokenmaxxing in" is clunky.
Suggested replacement (pairs with hed #1):
Big token bills are fine—as long as you can say what they bought. Here's how Every decides when an expensive AI experiment pays off, including one that burned through 4.5 billion tokens.
Shorter alternative:
Dan Shipper wants the team experimenting harder with AI. The catch: Every big run has to answer what it cost, what it bought, and what we learned.
Opening paragraph
What works: the three-times gap is a great fact, and "whether the rest of us are being ambitious enough" is the right note to end on before you pivot to "The goal isn't tokenmaxxing."
Problems:
- The first sentence is the weakest one. "Often if you are … you might think about…" hedges twice, uses a generic "you," and is missing a comma after "company." The paragraph builds up to its best fact instead of starting with it.
- It makes the same point twice with different levels of certainty. Sentence two says Dan thinks everyone should "consider ramping up." Sentence four says the gap has him wondering whether people are ambitious enough. Keep one. The "wondering whether" version is more accurate and makes the better closer.
- It drops a factual qualifier. The leaderboard is specifically OpenAI token usage (per the caption), but the lede says only "tokens." Say "OpenAI tokens" so the "more than three times" claim stays exact.
- The pronouns shift. It moves from "you" to "the rest of us." Since this is an Inside Every piece, commit to "us."
- "Every CEO Dan Shipper" is fine mid-sentence, but "Every's CEO, Dan Shipper" or "Dan Shipper, Every's CEO" avoids a stumble on "every CEO."
Suggested replacement (keeps all the qualifiers):
Most people who find themselves at the top of their company's spending chart take it as a cue to cut back. Dan Shipper, Every's CEO, took it as a cue to look at everyone else. He uses more than three times as many OpenAI tokens as the next-highest user at the company, and the size of that gap has him wondering whether the rest of us are being ambitious enough with AI.
Tighter alternative that leads with the fact:
Dan Shipper uses more than three times as many OpenAI tokens as anyone else at Every. Most people would see that as a reason to rein themselves in. Dan sees it as a reason to wonder whether the rest of us are being ambitious enough with AI.
Either version flows directly from hed #1 and sets up paragraph two ("The goal isn't tokenmaxxing…") without changes.
Recommended package
- HED: Our CEO Thinks We're Not Spending Enough on AI
- DEK: Big token bills are fine—as long as you can say what they bought. Here's how Every decides when an expensive AI experiment pays off, including one that burned through 4.5 billion tokens.
- LEDE: the first suggested replacement above.
Assumption: I treated "Token Math" as a possible recurring column name. If it isn't one, drop it entirely.
Checks
7/12ai-wordingevery task
deliverable text contains none of the shared AI-tell phrases in bench/ai-tells.txt. Task checks may add narrower contractual bans, but must not repeat this shared floor.
inspected ANSWER.md
Q1S02b. Interesting headline: replacementthis case
Judge's reasoning
The first-choice option 'Our CEO Thinks We're Not Spending Enough on AI' only reports an internal directive, much like the rubric's failing example, and 'Spend More Tokens. Then Show Your Work.' has a second clause that adds little specific to this article.
▸Rubric
S02b. Does the AI’s feedback propose better headline options that all capture the interesting provocation in a concise, punchy way that would fit in an email subject line (at least mostly)? PASS if: The feedback actually proposes at least one replacement headline rather than merely recommending improvement, and every headline it proposes as an option meets the bar. This applies to recommended choices, ranked options, alternates and unranked lists. Options count even if they are not labeled usable; calling an alternate weaker or less punchy does not exempt it. A stronger option cannot rescue a weak proposed option. Clearly rejected examples are excluded. Each proposed option gives the intended Every reader a short, compelling, draft-supported angle, expressed concisely and punchily and mostly fitting in an email subject line. It may make the article’s argument about ambitious AI experimentation, foreground a specific experiment and what it taught us, or take another supported angle; no particular thesis wording, person or number is required. Each headline must give a reader outside Every a reason to care through a useful implication for their AI work, a compelling argument they can apply, or a vivid example with a transferable stake. Second person is not required, but it does help. The reader stake must be apparent from the headline itself; a dek cannot supply a missing reader stake. “The Case for a Bigger Token Bill” and “Why You Should Use More Tokens” are nonexhaustive passing directions; similar supported formulations pass without exact matching. Internal examples are allowed when they signal the larger argument; this is not a blanket ban on CEO or company anecdotes. If an option shifts the center away from the current Dan-led opening, the feedback must explicitly call for recentering the introduction and explain how, and supply a dek explicitly paired with that headline. The paired dek must stay on that option’s focal story or question and add a specific supported answer, consequence or stake. For a Randy 4.5-billion-token angle, it should tell what the run bought or taught, such as a benchmark and a leaner agent setup; merely pivoting to general spending decisions or sharing the words tokens or experiment does not continue that promise. No particular dek wording is required. In a two-part headline, the second part must tell the reader something specific about this article’s angle, question or payoff. “Spend the Tokens, Then Do the Math” fails the interest bar: the idiom is comprehensible, but only says evaluation will happen afterward and adds no information about what this article asks or reveals. Do not fail an idiom merely because it is an idiom; assess its information in this context, separately from S02c comprehension. Keep hedging and detailed qualifications out of the headline. Essential identifying context needed to understand the subject, action, or comparison is allowed and is not a qualification; boldness does not require ambiguity. FAIL if: The feedback proposes no replacement, or any headline it offers as an option fails the bar: proposed options remain topic labels, generic spending/value claims or questions, isolated facts, unsupported broad claims, or weak side stories without an explicit intro-recentering plan and an explicitly paired dek that continues their particular promise; only report an internal directive without an apparent stake for an outside reader; add an empty second clause that tells the reader nothing specific about this article; bury the claim in qualification; or sprawl beyond mostly fitting in an email subject line. “Our CEO Wants Us to Use More Tokens” fails this reader-interest bar because it only reports an internal directive; basic clarity and topicality are insufficient. Do not impose an invented exact character limit. Recognizing the original problem alone does not pass this replacement check. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q2S03. Promise cashed outthis case
Judge's reasoning
The recommended headline 'Our CEO Thinks We're Not Spending Enough on AI' presents the draft as a workplace story about a CEO directive rather than a broader case for ambitious experiments.
▸Rubric
S03. Does the AI’s feedback assess whether the headline/subhead combination makes a promise that the article cashes out, using a loose standard? PASS if: The feedback considers whether the pair’s interest, answer or payoff is broadly delivered by the article, and any replacement pair it offers promises a broadly supported payoff. Equivalent discussion of framing accuracy, payoff or reader expectations counts. The body can supply nuance and qualifications; the headline need not mirror it literally. Qualitative returns such as learning, reusable benchmarks and better workflows can cash out an experimentation/ROI promise. A numerical ROI calculation is not required. FAIL if: The feedback ignores the promise/payoff question, recommends a promise the article does not come close to delivering, or insists on headline qualifications for literal faithfulness. Do not fail merely because the original article offers qualitative rather than numerical returns. The token maxing backlash is wrong remains borderline rather than an unconditional pass or fail. Also FAIL if an offered headline makes a supporting detail look like the article’s central story and leads the reader to expect a materially different kind of article, even when the literal claim appears in the body. “Our CEO Wants Us to Use More Tokens” frames this draft as a workplace story about how employees respond to a CEO directive; the draft uses his usage gap to make a broader case for ambitious AI experiments and examining what each experiment produced. A dek cannot fully rescue a headline whose primary frame points at the wrong story. Do not fail every headline that mentions Dan or Every: a headline can and should use internal examples when they accurately signal the larger argument. This does not require a full article summary or detailed headline qualifications. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q3S06. Point lands quicklythis case
Judge's reasoning
It flags double hedging and says the paragraph 'builds up to its best fact' but never identifies that the first sentence's own point comes at its end, and its first replacement keeps the same 'top of their company's spending chart take it as a cue to cut back' setup.
▸Rubric
S06. Does the AI’s feedback notice that I only know the point of the first sentence—“reining yourself in”—at the very end? PASS if: The feedback specifically identifies the first sentence’s delayed point or condition-heavy setup and recommends getting to its point sooner or replacing that framing entirely. Equivalent descriptions count. This is sentence structure, distinct from where the paragraph introduces the statistic. FAIL if: The feedback only says the paragraph takes too long to reach the statistic, or vaguely calls the opening weak, without recognizing the first sentence’s delayed point. Also fail a proposed fix that keeps the same delayed setup. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q4S07. Interesting setup immediatelythis case
Judge's reasoning
Both replacement openings delay the ambition implication behind a familiar reaction ('take it as a cue to cut back' / 'Most people would see that as a reason to rein themselves in') instead of presenting it right away.
▸Rubric
S07. Does the AI’s feedback identify the interesting gap and what it implies, and recommend setting that up really quickly—in the first sentence, or at least most of it in the first sentence? PASS if: The feedback identifies the surprising possibility that the rest of the team is not being ambitious enough and calls for presenting it immediately. Any replacement opening it offers delivers that direction: it can combine the implication with the gap or lead with a provocation whose supporting gap follows promptly. In a gap-led version, connect the ambition implication in the first sentence or the immediately following sentence. Dan’s If your team is not wasting tokens, there’s probably something wrong is a passing provocation-first direction; numerical evidence can follow. Preserve the source’s more-than-three-times comparison when stating the number and the uncertainty of the inference. FAIL if: The feedback misses the gap’s ambition implication, leaves it buried, or recommends an opening that leads with a familiar spending warning or states the statistic and delays its interesting implication behind another familiar reaction. Do not require both number and implication in one literal sentence or exact wording from an example. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q5S08. Cutback setup fits the CEO casethis case
Judge's reasoning
Both offered openings rely on the unsupported default that the top spender would cut back: 'take it as a cue to cut back' and 'Most people would see that as a reason to rein themselves in.'
▸Rubric
S08. Does the AI’s feedback avoid an ill-fitting default cutback reaction in every offered replacement opening for this CEO case? PASS if: Every usable offered opening avoids treating an imagined order or default expectation to cut back as the normal reaction to this CEO’s token spending. Dan: “usually the biggest spender is asked to cut back” does not typically apply when the biggest spender is the CEO, and is familiar/boring. A supported description of Dan himself weighing whether to spend less, or Arielle’s actual card/leaderboard process, may pass. FAIL if: Any usable offered opening relies on the unsupported biggest-spender-usually-asked-to-cut-back contrast as the hook for Dan’s CEO case, including equivalent unsupported defaults such as the obvious move is to rein it in or cue to cut back. Fail even if the feedback elsewhere diagnoses the flaw; a good alternative does not rescue a usable bad opening. A quote used solely to criticize the wording, or a clearly rejected example, is not an offered opening. This tests contextual fit, separately from S07’s sentence-placement rule. If no replacement opening is offered, do not fail this check solely for the omission. Return PASS or FAIL and one sentence citing the offered setup or absence of such a setup.
Q6S02c. Headline makes sense on its ownthis task
Judge's reasoning
All three suggested headlines stand on their own: 'Our CEO Thinks We're Not Spending Enough on AI' states a clear claim, 'Spend More Tokens. Then Show Your Work.' is a clear pair of instructions, and 'What 4.5 Billion Tokens Bought Us' clearly names a spend and asks what it produced, so none needs the dek to fill in a missing meaning.
▸Rubric
S02c. Does each headline offered as a usable replacement make sense on its own to the intended reader? PASS if: Each proposed headline communicates an intelligible claim, question, or subject without requiring the dek, article, or accompanying explanation to supply missing meaning. The subject, action, referents, and any comparison or scope essential to understanding are clear enough that the reader can restate the intended meaning without guessing or filling in an unstated premise. Recognizing a topic or familiar phrase is not enough when the central action or relationship remains unspecified. The reader must understand what is being acted on and what any linked evaluation, comparison, or consequence refers to; a plausible interpretation supplied by the reviewer is insufficient. Normal audience knowledge and familiar vocabulary are allowed. FAIL if: Any headline offered as usable depends on an opaque slogan, an unspecified action or referent, or omitted context essential to understanding. A stronger alternative or explanatory dek does not rescue that headline. Clearly rejected examples are excluded. A headline may withhold an answer to create curiosity, but its premise must be understandable. Essential context is different from hedging or detailed qualifications; do not require a complete summary or every factual qualification. Assess clarity here, separately from interest and factual support. If no replacement headline is offered, do not fail this check for the omission; S02b assesses that. Return PASS or FAIL and one sentence of evidence from the proposed headline copy.
Q7S01. On topicthis case
Judge's reasoning
The feedback treats 'Token Math' as an on-topic label and says the piece is about spending freely and then accounting for it, and its main replacements stay on token spending and ambitious experimentation.
▸Rubric
S01. Does the AI’s feedback assess whether the headline talks about the main thing in the article? PASS if: The feedback correctly treats Token Math as on topic and relates the framing to the article’s main subject or idea: token spending, ambitious AI experimentation and what experimentation buys or teaches the team. Any replacement headline stays on that main thing. Interest is assessed separately in S02a and S02b. FAIL if: The feedback wrongly calls the original headline off topic, misses the article’s main subject or proposes a replacement headline about a tangent. Do not fail merely because the original headline is on topic but uninteresting. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q8S02a. Interesting headline: diagnosisthis case
Judge's reasoning
It says '"Token Math" is a label, not a claim' and identifies the surprising idea it misses: 'The CEO sits at the top of the spending chart and thinks everyone else is underspending.'
▸Rubric
S02a. Does the AI’s feedback identify why the headline isn’t interesting enough—why it fails to capture the most interesting, counterintuitive or surprising part of the article that makes me feel, “ooh, I need to read that”? PASS if: The feedback recognizes that Token Math merely names the topic and does not convey the article’s interesting provocation, tension or payoff. Equivalent descriptions count; it need not use Dan’s exact words. This check assesses recognition, so no replacement is required to pass it. FAIL if: The feedback misses the interest problem, treats being on topic as sufficient, or only offers vague advice without identifying what is missing. Do not fail this diagnosis check because a replacement headline is weak; assess that separately in S02b. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q9S04a. Headline and dek connect: diagnosisthis case
Judge's reasoning
It says the dek 'is a second question stacked on the headline. It doesn't add a detail or an implication.'
▸Rubric
S04a. Does the AI’s feedback identify whether the headline and subhead form a connected whole, where the headline raises a question or topic of interest that the dek deepens and gives more details on? PASS if: The feedback recognizes that Token Math and What’s the ROI on experimentation? relate but their connection is not obvious, or gives equivalent criticism that the dek does not clearly develop the headline. This check assesses recognition, so no replacement pair is required to pass it. FAIL if: The feedback reviews the two in isolation without addressing their relationship, treats the original pairing as fine, or misses the difference between being loosely related and forming a connected whole. A good replacement pair alone does not establish that the feedback identified this problem; assess its copy separately in S04b. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q10S04b. Headline and dek connect: replacementthis case
Judge's reasoning
It pairs 'Our CEO Thinks We're Not Spending Enough on AI' with 'Big token bills are fine—as long as you can say what they bought…', a dek that develops the headline with a condition and a mechanism.
▸Rubric
S04b. Does the AI’s feedback propose a headline/subhead pair that actually forms a connected whole, with the dek deepening the headline and giving more details on it? PASS if: The feedback actually proposes at least one usable headline/subhead pair. The connection is clear and the dek develops the headline’s question or claim with an implication, mechanism, stake or detail in concise, compelling language. A headline and dek recommended together can count without a prescribed presentation format. For Why you should be spending more on tokens, Maximizing your company’s AI use requires risky experiments illustrates the relationship. Exact wording or a risk theme is not required. Judge the relationship here; headline interest is assessed separately in S02b. FAIL if: The feedback proposes no usable pair, or none of its proposed pairs clearly develops the headline: the dek is unrelated, merely restates it, leaves its connection unclear, or introduces a different promise. Merely identifying the original problem does not pass this replacement check. The spoken four-X-engineers example illustrates structure; it is not an established source fact or grading target. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.
Q11S05. Weasel wordsthis case
Judge's reasoning
It says '"Often if you are … you might think about…" hedges twice', and the replacement openings drop that hedging.
▸Rubric
S05. Does the AI’s feedback call out weasel words in the headline, dek, or first line, like “often” and “you might think about”, that make the point tentative? PASS if: The feedback identifies unnecessary hedging, tentativeness or needless indirection in the original lead-in, tied to an actual phrase such as often or you might think about. Equivalent wording counts; it need not say weasel words or name both examples. Where it offers replacement copy, that copy removes the empty lead-in. Preserve uncertainty that carries real meaning: possible, probably, might or similar words can be appropriate for a genuine inference. FAIL if: The feedback ignores the unnecessarily tentative lead-in, discusses only a different issue, or repeats equivalent empty hedging in its replacement. Do not apply a blanket ban on uncertainty words. Return PASS or FAIL and one sentence of evidence from the AI’s feedback.