04 thematic

Northcove AI engineering training: what to focus on

Train the engineering org on four things, in this order:

1. Company-specific context — rules, examples, and what to leave out of a prompt.
2. Verification matched to risk — a different bar for throwaway work and for production, money, and regulated data.
3. The daily Cursor workflow — plan, then small diffs, with concrete examples from this codebase.
4. Eval discipline for teams already shipping LLM features — a smaller, separate track.

The first session should leave behind shared artifacts (rules, a review checklist, a context-pack example), not only a recording. Northcove’s models and Cursor chats fail on this codebase and this compliance bar. A generic “use AI better” workshop will not move the 100-person org.

Assumption: this is a first program for engineering (Consumer, Business, Platform), time-boxed to about a day of core workshop plus a short follow-on for LLM-feature teams. Format and date were not specified. The sample is three conversations: Nina (full-stack, basic Cursor user, shipping a document-interpretation feature), Ben (senior, Business, power user, correctness-critical money logic), and Raj (org view). Consumer is represented only through Raj’s overview.

Where the three conversations agree

Northcove is already on Cursor under an enterprise license. Tab-complete is normal. A slice of the org, including Raj’s team, now edits more than it authors. Most of engineering is still AI-second: AI after the real work starts, not as the default way to plan, draft, and check. Senior engineers with 4–10 years of experience are the skeptical center of gravity, and the codebase gives them reasons. The monolith carries startup-era coupling, hidden side effects, and antipatterns. Fast CI is 20–30 minutes and deploys run 40 minutes to 2 hours, so a plausible-but-wrong change is expensive. A prior third-party data-sharing incident makes regulated-data exposure a live constraint, not a policy slide.

The same gap shows up in three voices:

• Nina cannot feed scattered, stale docs (Google Docs, Confluence, Slack) into Cursor. A thin system prompt names the Python/TypeScript stack and then runs out. Complex prompts over-engineer. Underspecified prompts copy legacy patterns. Results improve when she pastes a similar example from the repo.
• Ben’s line is “context is king”: tag the relevant files, or the model cannot see the knob that breaks something else. He will not use AI for production features in multi-party balance logic for that reason. He uses it constantly for sketches, plans, debugging, and one-off scripts, where directionally correct is enough. He wants organizational rules, commands, and reusable prompts, and he has not gotten them.
• Raj’s confidence ladder matches both of them. Codegen with human review works today. Northcove-specific implementations do not, unless the prompt is very prescriptive. Non-trivial features, database migrations, and regulated-data work stay skeptical. The largest structural gap he names is siloed, stale knowledge.

Nina’s document model (Anthropic, under an organizational data-isolation contract, with a hand-labeled set of 100–250 documents) is a second job sitting next to “engineers using Cursor.” It needs its own module. It should not set the agenda for all 100 engineers.

1. Company-specific context

This is the highest-leverage focus. Every failure mode in the notes is a context failure: missing standards, stale docs pulled in as if they were current, legacy patterns treated as examples, and no shared place where Northcove’s rules live.

Teach a practical context pack, not prompt theory:

• Name the files that matter before asking for the change. Ben already does this; most of the org does not.
• Prefer a concrete example from the current code over a general instruction. That is the difference Nina sees between useful scaffolding and a wrong cherry-pick.
• Encode stable standards in shared Cursor rules: Python and TypeScript conventions, framework choices, test layout, and a short list of patterns that are known antipatterns in this monolith.
• Keep the pack small. A rule that tries to restate all of Confluence will go stale the way the existing docs already have. Point at the living source (the repo, a current runbook) and quote only the constraint the model keeps violating.
• Treat “which docs are safe to attach” as part of the skill. Slack threads and old Google Docs are often wrong.

Session output: each squad leaves with a starter rules file seeded from real review comments, plus one worked example of a context pack for a task they actually have in flight. Business (Ben’s world) and Platform should each contribute rules; a single generic file will miss the coupling he described.

Ben asked for org buy-in on rules, commands, and reusable prompts. Assign an owner before the workshop ends. Unowned rules will join the stale-doc pile Raj described.

2. Verification matched to risk

Adoption is splitting by risk tolerance, and the training should make that split explicit so seniors recognize their own standard in it.

Two speeds, taught as judgment:

• Directionally correct. Internal scripts, throwaway prototypes, planning sketches, UX mockups in the spirit of the Vercel v0 flow product already uses. Ben’s bar: it does not need to compile if the point is alignment. Cheap to throw away. This is where AI-first should be the default, including for engineers who refuse AI on production paths.
• Correctness-critical. Anything that decides what a user owes, touches Stripe, invoicing, recoups, schema, or regulated data. Raj’s bar: human review, especially around databases and regulated data. Nina’s failure mode belongs here: output that looks finished and carries a subtle bug, accepted because the team lacks end-to-end and integration coverage.

Teach the review habits that make the second speed acceptable to a skeptical senior:

• Tests or an explicit check before accepting the diff. Writing tests into an existing suite is already one of Nina’s reliable uses; make it the default close of an AI change, not an optional follow-up.
• Small diffs. The monolith’s hidden side effects are why Ben keeps production features in his own hands.
• A short checklist for “plausible but wrong”: behavior at the boundary the prompt did not mention, a pattern copied from legacy code, a migration or query the author did not read line by line.
• Data boundaries. What may enter Cursor or a model prompt, what stays inside the isolation contract Nina’s feature already uses, and what a prior data-sharing incident put off limits. Legal and compliance already sit inside project decisions; the workshop should use Northcove’s actual rules, not a generic AI-policy deck.

Session output: a one-page review checklist posted where PRs happen, with the two speeds labeled in Northcove’s language (loss per transaction, processing accuracy, timely recoups — the metrics Ben’s group actually moves).

The editor-not-author shift Raj described makes this module urgent. If people rarely type more than about 20 lines themselves, the skill that matters is review. Training that stops at generation will scale the “good enough” risk Nina named.

3. The daily workflow

The skill gap inside engineering is wide. Nina calls herself a basic user. Ben never rough-drafts by hand and is starting to point agents at test coverage, flag removal, and debugging. Everyone else is somewhere on that line, and most are closer to tab-complete plus occasional chat. Raj already heard the Cursor-recommended loop — plan in chat, then build — and code search is how people learn an unfamiliar corner of the monolith. Teach that loop until it is boring.

Workshop drills, on a Northcove repo, not a toy app:

• Plan in chat against real files, then implement only the planned slice.
• Ask for the initial structure and the tests, then edit. Nina’s reliable cases are scaffolding and tests, not “build the feature.”
• Change one increment. Her back-and-forth cost comes from prompts that are too large and from re-prompting when a small wording change moves the result.
• When the model over-engineers, narrow the prompt and attach an example. Show this failure live; it is the one she can already name.
• Use the agent on a bounded chore with a clear done state: extend a test suite, delete a dead flag, explain a failing CI run. Ben’s early agent wins are all of this shape.

Defer unbounded “go build the feature” agent demos. All three are aligned that large, weakly specified features fail here: Nina wants them and does not yet have the prompting for them, Ben will not put them on production business logic, Raj is skeptical of significant non-trivial features. Agents become a later office-hours topic once rules and the review checklist exist. Subagents, MCP servers, and automated prompt optimizers (DSPy) can wait. Ben already tried MCP beyond the database, hit performance problems, and went back to CLI tools the model already knows. That is a useful aside in office hours, not a module.

Session output: a one-page “how we use Cursor here” that matches what power users already do, written so a basic user can follow it tomorrow: plan, attach examples, small diff, tests, human review.

4. Eval discipline for LLM features

Keep this off the all-engineers agenda. Put it in a follow-on for teams building model-powered product, with Nina’s document-interpretation work as the case.

What that team is already doing, and should be taught as a repeatable loop:

• A ground-truth set. They hand-labeled on the order of 100–250 documents because layouts and terminology differ by source. Teach how big a set needs to be for the decision in front of them, and how to add cases when a new source appears, so labeling does not stay a one-time hero project.
• A check that runs when the prompt or model changes. Small prompt edits move results in non-deterministic ways, and the current loop is more labor-heavy than classical ML training. The recent move toward proxy metrics at larger scale is the right evolution; teach how to pick a proxy that still tracks the labeled set, and when the proxy is lying.
• Data isolation as a design constraint from the first prototype, which her v2 already inherited.

Skip automated prompt-optimization frameworks until a team has an eval set it trusts. Ben knows the idea and has not had time; the missing piece is the dataset and the habit, not the optimizer.

Session output: a template eval note — task, labeled set, metric, proxy metric, what triggers a re-run — that another squad can copy for the next extraction or classification feature.

Who sits in which room

Room	Who	Focus
Core workshop (all engineers)	~100 engineers, both suspicious seniors and newer full-stack	Context rules, the two-speed review bar, plan-then-small-diff
Working session inside the core day	One senior from Business, Platform, and Consumer	Draft the shared rules from real review comments
Follow-on (half day)	Squads shipping LLM features	Evals, prompt-change discipline, isolation contracts
Later office hours	People already using agents (Ben’s channel, not the whole org)	Bounded background agents: tests, flag removal, debugging

Run the core workshop in mixed groups so power users demonstrate on real tasks and basic users practice the same loop. A single demo from the Cursor power-user Slack channel will wash over Nina’s half of the org.

PMs, the Cursor Slack integration, and BI access are real demand — Ben sees non-engineers getting value, and product already prototypes in v0 — but they are a different session. Engineering training should not be redesigned around copy edits and roadmap docs. PRD drafting can wait with it: Raj says it could work and also says product workflows are why it is barely used, while engineering already spends its own time scoping because PRDs arrive thin.

What to leave for later

These came up, and they are the wrong center for the first program:

• MCP setup and subagent fleets. Early, uneven, and already disappointing on Confluence, Jira, and GitHub.
• Autonomous delivery of large features. The codebase and the compliance bar do not support it as a training outcome.
• Database migrations. Raj flagged them as infra pain and as work he does not want a model owning.
• User-research synthesis. Volume is a handful of interviews, which is below the threshold he would trust.
• A company knowledge-base project. The pain is real (Raj’s “major gap”), and the fix that belongs in this training is a small rules layer plus a habit of citing the repo. A migration of Confluence, Slack, and Google Docs is a separate program and will rot again if it is the workshop.

How to tell whether it worked

Check behavior a month later, on the back of the artifacts:

• Several squads have a rules file someone other than its author has edited.
• A sample of AI-assisted PRs in Business shows the checklist (tests or an explicit waiver, small diff, no unlabeled regulated data).
• Internal scripts and prototypes are drafted AI-first without a new debate each time. That is the AI-second to AI-first shift Ben described, aimed where he says it is safe.
• The document-interpretation team re-runs a named eval when the prompt changes, and can point to the proxy metric they now trust.

If those four are moving, a later session on background agents has something solid to stand on. If they are not, more tool surface area will reproduce the current split: a few power users, a suspicious majority, and context that still lives in people’s heads.