Scoreboard

6 models6 tasks

How scores work

Dan’s Editorial Checks is one person's benchmark: real work they use AI for, scored by the things they check when deciding whether an output is good. It answers "which model is best at my job, for the things I care about", not a general claim about model quality.

One row per model, on the setup (tool and effort) it ran last: each case is scored by the mean of that setup's public runs. A cell that counts several runs of one case says how many (×4) and the range from the lowest to the highest (60–85%); hover it for the standard deviation (SD, in points). Under each score a bar is ink for the score and hatched for the rest; the best score in each column is bold.

A run reads as checks passed over checks applicable. Code checks are answered by a program, model checks by an LLM judge, which is always Claude. A check counts once unless the benchmark's owner gave it a weight, said beside its name on a run: ×3 counts as much as three checks. A score a weight moved says its percent is by weight, because the fraction beside it still counts checks. A case can be run many times, and every run counts, averaged; a task is the mean of its cases, and Overall is the mean across tasks. Overall's bar is as long as the share of the tasks the model ran, and the rest of its track is empty. A cell opens the newest of the runs it counts: read the actual output beside the checks it passed and failed.

JSON feed