# AI Engineering Training Focus for Northcove **Source:** Meeting notes with Nina Park (full-stack, Consumer-adjacent product work), Ben Walsh (senior engineer, Business group), and Raj (engineering-org vantage). **Company snapshot:** ~100 engineers across Consumer / Business / Platform; Python + TypeScript monolith; Cursor enterprise already in place; strong IC autonomy; legal/compliance is a real constraint, not a checkbox. --- ## What to focus on (short version) Do **not** train this org on “how to use Cursor” or generic LLM hype. They already have licenses, tab-complete is popular, and a subset of people are power users. The bottleneck is **Northcove-specific context, verification, and judgment** — getting AI work that is safe to ship in a coupled, correctness-critical, compliance-sensitive codebase. Ranked focus: 1. **Context engineering for this codebase** (highest leverage; all three interviews) 2. **Review, testing, and “when not to ship AI code”** (the production gap) 3. **Prompting for incremental change, not greenfield** (Nina’s daily pain; Ben’s agent work) 4. **Safe AI in a regulated environment** (Raj + prior incident) 5. **Evaluating LLM features in product** (Nina’s document-interpretation work; Ben’s DSPy interest) 6. **AI-first workflow and shared org assets** (close the power-user / rest-of-org gap) Spend little training time on: MCP server setup, v0/Figma prototyping (already working), writing more Confluence, or “agents will replace scoping.” Those either already work, fail for known reasons, or need process/product changes more than a class. --- ## Current state (so training meets people where they are) | Pattern | Evidence | Training implication | |---|---|---| | Cursor is table stakes; agents are not | Everyone on Cursor; tab-complete popular; chat used for plan-then-build; agents only among early adopters | Assume tool familiarity. Teach *workflows*, not product tours. | | Org is still “AI-second” | Ben: most of eng is AI-second; he never drafts by hand. Raj: people are editors, rarely write >20 lines. Nina: self-described basic user vs Ben as power user | Training must convert AI-second → AI-first *with* guardrails, not just show more features. | | AI is used for sketches, not production | Ben avoids AI on real features because of coupling/hidden side effects. Nina: plausible-looking code with subtle bugs | The curriculum’s core skill is *production-grade AI use*, not demo-grade generation. | | Context is the shared failure mode | Stale docs in Docs/Confluence/Slack; no way to feed them to Cursor; AI cherry-picks legacy antipatterns; not good at Northcove-specific implementations | Teach how to *supply* context (files, examples, rules). Also build the reusable rules/prompts the org currently lacks. | | Experience split | More experienced engineers more suspicious; seniors drowning in meetings; juniors/mid may over-trust output | Two tracks in the same room: skeptics get verification/compliance; enthusiasts get context + review discipline. | | Compliance is load-bearing | Anthropic with data-isolation contracts; prior third-party data-sharing incident; caution on DB/regulated data | Every module needs a “what must not leave the boundary” rule, not a side slide. | --- ## Priority 1 — Context engineering for Northcove **Why this is #1.** Ben’s first teaching principle is “context is king.” Nina gets over-engineering and wrong legacy patterns unless she provides similar examples. Raj: AI is not good at Northcove-specific implementations and needs very prescriptive custom rules. This is the same problem stated three ways. **Teach:** - **Front-load files, not wishes.** Tag the relevant modules, tests, and a *good* example of the pattern you want — do not prompt “do the thing.” - **Examples beat general prompts.** Nina’s working method: concrete codebase examples → better results. Make this the default, not a trick. - **Good vs. bad context.** How to stop the model from cloning antipatterns, coupling, and hidden side effects from a fast-startup monolith. - **Rules, commands, and reusable prompts as org infrastructure.** Ben: quality requires standardized Northcove-specific prompts and guidelines, and org buy-in to actually use them. Nina already has a thin system prompt (Python/framework) and cannot encode standards because standards are incomplete. - **CLI over flaky MCP for now.** Ben tried Confluence/Jira/GitHub MCP, hit performance issues, and prefers CLI tools that are in the training set. Train the path that works today; treat MCP as optional, not the curriculum. **Do in the room:** a live exercise on a real (or realistic) Northcove-shaped task: “add a small billing/intake change.” First with a vague prompt, then with tagged files + a golden example + a short rule. Compare diffs. **Org work that training cannot skip:** a small, maintained set of Cursor rules / commands for Python + TypeScript house style, “never do X” antipatterns, and how to touch the monolith. Without that, training evaporates in a week. --- ## Priority 2 — Review, testing, and when not to ship **Why.** The org is already generating more code than it is authoring. Nina flags plausible-but-wrong output and weak e2e/integration coverage. Ben will not put AI on production features because one knob knocks another over. Raj is confident on assisted codegen *with human review*, skeptical on non-trivial features, migrations, and regulated-data work. If training only makes people generate faster, it increases the “good enough” risk Nina named. **Teach:** - **Editor mindset with a checklist.** What to look for in AI diffs: hidden side effects, coupling, missed invariants in multi-party money math, test gaps, copy-pasted legacy patterns. - **Tests as the contract you give the model.** Writing/extending tests *before or with* the change (Nina already uses Cursor for existing suites). How to ask for tests that match the *existing* harness, not a new framework. - **Where AI is in-bounds vs. out-of-bounds today** (make this explicit, not vibes): - **In-bounds:** scaffolding, tests against existing suites, debugging, throwaway sketches, internal scripts, feature-flag removal, coverage, planning. - **Out-of-bounds or dual-control:** database migrations, regulated-data paths, correctness-critical financial logic without a human who owns the invariant, large features without a plan + tests. - **Integration/e2e and monitoring as the missing layer.** Nina: lack of robust e2e/integration tests and validation. Training should show *how to ask the agent to add the test you actually run in CI*, and how to treat CI (20–30 min) / deploy (40 min–2 hr) as the real feedback loop — not the chat window. This is also how you bring skeptical senior engineers in: the class agrees with them that unreviewed AI in billing/intake is reckless, then gives them a faster *reviewed* loop. --- ## Priority 3 — Incremental change and background agents **Why.** Nina’s workflow is back-and-forth and time-consuming; she wants better prompting for *incremental* changes and is interested in long-running background agents. Ben is already using agents for coverage, flag removal, and debugging, but subagents are not core yet. Raj: agents are less adopted. The failure mode to train against is Nina’s: large, complex prompts → over-engineered PRs. **Teach:** - **Plan in chat, then implement a thin slice.** Matches the Cursor-recommended workflow Raj cited. Spec the change, get a plan, then generate *one* increment with tests. - **Prompting for diffs, not rewrites.** “Change this function in the style of file X; do not introduce new abstractions.” - **Background agents for well-bounded chores first:** test coverage, flag removal, reproducing a failure, draft docs from the PR. Not “build the feature while I go to lunch” on the monolith. - **When to take the wheel.** If the agent is rummaging through legacy code or expanding scope, stop and add context / shrink the task. Defer “fully independent feature agents” as an advanced optional lab. The monolith’s coupling makes that a reliability problem, not a prompting skill issue. --- ## Priority 4 — Compliance-safe AI use **Why.** Raj put legal/compliance on every decision. There was a prior third-party data-sharing incident that forced a rework of paid/organic surfaces. Nina’s document-interpretation feature already uses Anthropic under organizational data-isolation contracts. People are (correctly) skeptical of sending regulated data to models. If this is a footnote, cautious seniors will ignore the rest of the training, and enthusiastic users will recreate the last incident. **Teach (short, mandatory, scenario-based):** - What is allowed in Cursor/chat vs. what stays in isolated, contracted model paths. - No regulated payloads, customer documents, or production data dumps into general chat. - Database and PII-adjacent work: human-owned, extra review, often no AI on the data itself. - How to use AI on *structure* (schema change plan, test scaffolding) without pasting *contents*. - Who to ask when unsure (name the real owner; assume Legal/Security + a platform contact). Assumption: enterprise Cursor is configured with org data protections similar to the Anthropic isolation Nina described. If that is not true, fix configuration *before* a broad training push. --- ## Priority 5 — Evaluating LLM features (for people building them) **Why.** This is a different skill from “AI as coding assistant.” Nina is already shipping LLM-powered document interpretation: messy layouts, varying terminology, manual labels on 100–250 docs, painful prompt iteration, non-determinism from small prompt edits, recent move to proxy metrics. Ben knows DSPy-style prompt optimization would help many projects and has not had time. A subset of engineers (and PMs using v0) will keep putting models *in the product*. They need evaluation literacy, not more playground prompting. **Teach (a half-day track, not the whole offsite):** - Gold set vs. proxy metrics: when 100–250 hand labels are enough, when they lie, how to stop evaluating by vibe. - Prompt changes as experiments: one variable at a time; expect non-determinism; record traces. - What “better” means for document intake: error types users actually feel (wrong party, wrong amount, missed field), not BLEU-like scores. - Lightweight optimization: when to bother with systematic prompt optimization (DSPy or equivalent) vs. a better gold set. - Human review in the loop for correctness-critical extraction (fits Business-group stakes: what users owe). Do not make this the default track for all 100 engineers. Most need Priorities 1–4. --- ## Priority 6 — AI-first workflow and shared assets (close the Ben/Nina gap) **Why.** Ben uses AI for questions, reproduction, documentation, calendar/Jira, sketches, and internal tools. Nina is a basic Cursor-chat user. Raj notes knowledge is siloed and stale; seniors lose days to meetings. Non-eng (PMs) already hammer Cursor Slack for copy and change requests. Training should spread *habits and artifacts*, not just individual hero usage. **Teach:** - A default daily loop: reproduce → plan → implement slice → tests → PR summary, all with the same context pack. - Directionally-correct prototypes for alignment (Ben: people are visual; cheap to throw away). Pair with existing v0 use by PMs so eng and PM share a prototype language. - How to share what works: the `cursor-nerds` / `background-agents` Slack channels already exist — training should end with “publish one rule or command,” not “go try it.” - What not to stuff into tech specs. Ben: current specs try to solve strategy, dates, staffing, *and* technical alignment. AI can draft a technical plan or a prototype; it should not be used to inflate the same overloaded doc. **Non-eng note (optional add-on, not the eng core):** PMs and other functions are already in the tools. A short companion session on “how to ask eng-shaped questions / how v0 mockups become a real ticket” will reduce thrash. BI hookup is a platform project, not a training module. --- ## Suggested shape of the training Assume a **1.5–2 day** core for ICs, plus a **half-day** LLM-in-product elective, plus a **90-minute** lead/PM companion. Northcove’s culture is autonomous and meeting-fatigued; keep it studio-style on real work, not slide decks. ### Day 1 — Context + compliance + review (everyone) | Block | Outcome | |---|---| | 45 min | Shared picture: what is working (tab, chat-plan-build, sketches) vs. the production gap. No tool demo. | | 90 min | Context lab on a Northcove-like change. Vague prompt vs. tagged files + golden example + rule. | | 45 min | Compliance scenarios (documents, billing data, migrations). Pass/fail, not discussion theater. | | 90 min | Review lab: three AI-generated diffs, one of which is subtly wrong in business logic. Practice the editor checklist. | | 30 min | Install/adopt the starter Cursor rules pack (or write the first three rules live). | ### Day 2 — Incremental agents + workflow (everyone) | Block | Outcome | |---|---| | 90 min | Incremental-change prompting; stop over-engineering. Convert a “build X” prompt into a slice + tests. | | 75 min | Background agent lab on a *bounded* chore (coverage, flag removal, repro). When to kill the agent. | | 60 min | Personal workflow: pick one recurring task (debug, script, PR description, test) and AI-first it. | | 30 min | Publish one reusable prompt/rule/command to the existing Slack channels. | ### Elective (half day) — People shipping LLM features Gold sets, proxy metrics, non-determinism, isolation contracts, “do we need prompt optimization or more labels?” Use the document-interpretation system as the case, not a toy chatbot. ### Companion (90 min) — Tech leads / PMs When a prototype is enough to align; what belongs in a spec vs. a throwaway; how to staff review time as AI increases PR volume; do not add meetings to “drive AI adoption.” --- ## What not to spend the scarce calendar on - **Cursor 101 / tab-complete.** Already universal and valued. - **MCP as the knowledge-management fix.** Tried, slow, and the underlying docs are stale. Knowledge hygiene is an ops problem; training people to configure MCP will not un-stale Confluence. - **“Write better Confluence so the model can read it.”** Docs are already scattered and rotting. Teach attaching *the files that compile* and a short rule. A later platform effort can tackle search/BI. - **Fully autonomous feature development on the monolith.** Nina wants it; coupling and hidden side effects make it a poor first skill. Bounded agents first. - **PRD generation.** Raj: product-org workflows mean this is unused. Engineering already scopes; training PMs to dump PRDs into a model will not match how work starts (ops + marketing + a few days of discovery). - **User-research synthesis at scale.** Raj: they only talk to a few customers; volume is not there. v0 mockups for qualitative testing are already a strength — leave them alone. --- ## Audience split (same curriculum, different emphasis) | Audience | Tilt | |---|---| | **Nina-like (solid IC, basic AI user)** | Context with examples, incremental prompts, review checklist, “you can go faster without accepting plausible junk.” | | **Ben-like (power user, production-cautious)** | Help them productize what they already do: rules/commands, agent chores, teach-backs. Ask them to lead the review lab. Do not waste their time on chat basics. | | **Skeptical seniors (Raj’s 4–10 year cohort)** | Lead with compliance, financial correctness, and failure cases. Then show a faster *reviewed* loop. Do not sell “AI-first” as identity. | | **People building LLM product features** | Elective on evals, isolation, gold sets. | | **PMs / non-eng already in Cursor Slack** | Companion session only; keep eng training engineering. | Ben is the internal multiplier. Use him (and the existing `cursor-nerds` channel) as faculty, not as the target learner. --- ## Success in 60 days (so this is not a one-off) Training worked if: 1. A **short, official Cursor rules/commands pack** exists and is referenced in PRs (Python + TypeScript + “do not copy these antipatterns”). 2. **AI-generated production PRs** routinely include tests that match the existing suite, and reviewers use a shared checklist for subtle breaks. 3. **Agents are used on bounded chores** (coverage, flags, repro) by more than the current early-adopter set — and *not* used as a default for migrations or regulated-data changes. 4. **No new data-handling incidents**; people can answer “does this payload belong in chat?” without a Slack thread. 5. The **Ben–Nina gap narrows**: more ICs tag files and examples by default; power-user tricks are written down instead of trapped in one person’s session. If those artifacts are not funded, skip a large training offsite and do a 2-hour context+compliance studio plus the rules pack. The interviews are clear that *organizational* reusable context is the constraint, not motivation. --- ## Assumptions (none of these change the ranking) - Training is for Northcove engineering first; non-eng enablement is adjacent. - Cursor remains the standard; Claude-code experiments stay optional. - Enterprise tool configuration already aims at data isolation; if not, configuration precedes training. - “Business group” work (what users owe, invoicing, Stripe, intake) is representative of the highest-cost failure mode, even for Consumer/Platform engineers who should still learn the same review habits. - Headcount ~100, mostly 4–10 years’ experience, high autonomy, single-engineer projects common — so training must be usable without a new process bureaucracy. - I did not have recordings, raw transcripts, or an existing curriculum; synthesis is from the three interview writeups only. --- ## One-line brief for whoever owns the program **Train Northcove engineers to give models the right local context, change the monolith in small verified slices, and keep regulated data and correctness-critical logic under human control — then freeze that into shared rules so the org is not dependent on a few power users.**