# Northcove AI Engineering Training: What to Focus On *Synthesis of discovery interviews with Nina Park (full-stack), Ben Walsh (Senior SWE, Business), and Raj (org-level view: role not stated in the notes)* --- ## The short version Northcove doesn't need an "intro to AI coding" course. Everyone already uses Cursor. Tab-complete and chat are common, and Raj says engineers "rarely write more than 20 lines manually." The problem is that **AI stops helping at the point where Northcove's own context starts**: legacy coupling, company-specific patterns, regulated data and correctness-critical business logic. So engineers keep AI on the safe edges (scaffolding, tests, scripts, debugging) and keep production work mostly human-driven. The training should focus on three things, in this order: 1. **Context engineering for Northcove.** Teach people to give the AI the right context, and leave the training with a shared set of Northcove rules, commands and reusable prompts. 2. **Verification and risk-tiered use.** Teach when "directionally correct" is fine and when output needs strict checking. This is what earns trust from the skeptical senior engineers and from Legal and Compliance. 3. **Delegating to agents.** Move people from back-and-forth chat to scoped background-agent work they can hand off, starting with the tasks early adopters already trust agents with. Two optional tracks for smaller groups: **building and evaluating LLM features** (Nina's team, and anyone shipping LLM features) and **AI for planning and prototyping** (tech leads and PMs). --- ## Theme 1: Context is the main bottleneck (raised by all three) | Who | What they said | |---|---| | Nina | AI "cherry-picks wrong context from legacy code patterns." It does better when given similar examples than general prompts. There's no structured way to feed docs to Cursor. Her system prompt only covers basic Python/framework specs, and writing a full standards prompt is hard. | | Ben | "Context is king": tag the relevant files up front. He avoids AI for production features because of "antipatterns, coupling, hidden side effects… turn one knob, something else falls down." He says the org needs buy-in for **rules, commands, reusable prompts** and "standardized Northcove-specific prompts." | | Raj | AI is "not good at Northcove-specific implementations" and needs "very prescriptive custom rules." He calls siloed, stale knowledge (Confluence, Slack, docs) a **major gap**. | **What this means for training** - Teach **context loading as a skill**: tagging files, pointing at a *good* example ("make it like `X`, not like the legacy `Y`"), stating constraints and non-goals, and keeping prompts small so the model doesn't over-engineer (Nina's failure mode). - **Write the context down during the training, not just talk about it.** Run a hands-on session where each team drafts Cursor rules for its area: preferred patterns, banned legacy patterns, known coupling hazards ("if you touch X, also check Y"), and how to test. Check them into the repo so they get reviewed like code. This gives Ben the organizational buy-in he's asking for and gives Nina the standards prompt she couldn't write alone. - Cover **how to pull in outside knowledge**: CLI tools vs. MCP (Ben found MCP for Confluence/Jira/GitHub slow and prefers CLIs). Keep this practical. Show which approach works in Northcove's setup today. - **Scope note:** the scattered, stale knowledge problem across Confluence, Slack and Docs is bigger than training can fix. It's an infrastructure and ownership issue. Training can show how to write down knowledge the AI can use (rules files, short ADRs near the code). Fixing the knowledge base itself should be a separate recommendation to leadership. ## Theme 2: Verification, guardrails and compliance (raised by all three) | Who | What they said | |---|---| | Nina | "AI code looks plausible but may introduce subtle issues." There's a lack of end-to-end and integration testing, and a risk of accepting "good enough" output. | | Ben | His domain is **correctness-critical** multi-party money calculations (what users owe, invoicing, Stripe, refunds), and hidden side effects are everywhere. | | Raj | Codegen is "good today" but needs human review, *especially for database and regulated-data work*. He's skeptical of AI for migrations and big features. Regulated-data exposure is a real worry: a past third-party data-sharing incident forced rework, and litigation risk makes the org cautious. He also notes more experienced engineers are more suspicious of AI. | **What this means for training** - **Teach a risk-tier framework.** This resolves the one real tension in the interviews: Ben's "directionally correct is the name of the game" vs. Raj and Nina's caution. Both are right, for different work: - *Tier 1, low risk* (throwaway prototypes, one-off scripts, internal tools, spikes): AI-first. Directionally correct is enough. - *Tier 2, standard product code*: AI drafts. The engineer reviews against the rules and tests. - *Tier 3, high risk* (payments and money math, database migrations, regulated data, auth): AI can help plan, write tests and review, but requires clear acceptance criteria written up front, tests the human has checked, and extra review. - **Test-first habits with AI**: have the AI write characterization and integration tests *before* changing coupled code, and use AI to hunt for edge cases in business logic. This deals with Nina's worry about missing end-to-end and integration tests directly. - **Reviewing AI-written PRs**: what subtle AI errors look like (plausible but wrong edge cases, copied legacy anti-patterns, unnecessary abstraction), and how to use AI as a second reviewer. - **Data handling rules, co-presented with Legal and Compliance**: what data can go into which tools under the enterprise contracts (Cursor enterprise; Nina notes Anthropic organizational data-isolation agreements), what must never go in, and how to work on regulated-data code safely. Given the history, this module is needed for credibility with leadership and senior engineers. - **For the skeptics, use Northcove's own code**: run demos on the real monolith, including failures and how to catch them. Generic success demos will backfire with experienced engineers. ## Theme 3: Moving from chat to delegated agents (raised by all three) | Who | What they said | |---|---| | Nina | Her workflow is "very back-and-forth, time consuming." She wants background agents, better prompting for incremental changes, and to hand off larger features. She calls herself a "basic user." | | Ben | He's starting to use agents for test coverage, feature-flag removal and debugging. Subagents aren't core to his workflow yet. Most of the org is in "AI-second" mode. | | Raj | "Agents less adopted, some early adopters experimenting." Planning in chat, then building, is the current recommended pattern. | **What this means for training** - Teach **how to scope a task for an agent**: clear acceptance criteria, relevant files, a test command, a small blast radius. Start with the use cases early adopters already trust (test coverage, flag cleanup, dependency bumps, reproducing bugs), then move up to small features. - **Plan → execute → verify loops**: breaking a feature into small, checkable increments. This covers Nina's "incremental changes" request. - **Account for the 20–30 minute CI and 40 minute to 2 hour deploy.** Slow feedback makes batching and parallel background agents *more* useful, but engineers need local checks the agent can run itself, so it isn't waiting on CI. - **Use internal champions.** Ben and the existing `#cursor-nerds` and `#background-agents` Slack channels are an in-house network already. Have Ben (or similar power users) co-teach, and after the training turn the channels into a place to share prompts, rules and agent recipes. ## Theme 4 (optional track): Building and evaluating LLM features Nina's document-interpretation feature shows a separate skill set: hand-labeled ground truth (100–250 docs), prompt changes that shift results unpredictably, and a recent move to proxy metrics at larger scale. Ben mentions DSPy and says "many projects would benefit from automated prompt optimization." **Suggested content:** designing eval sets, regression-testing prompt changes, handling non-determinism (repeated runs, tolerance thresholds), LLM-as-judge with checks against human labels, proxy metrics in production, and a short practical look at automated prompt optimization (DSPy or similar). Offer this to the engineers shipping LLM features, not the whole org of 100. ## Theme 5 (optional track): AI for planning, specs and prototyping - Ben says tech specs are "too much": they try to cover strategy, deadlines, resourcing, sequencing and technical design in one document. He proposes a quarterly onsite that uses AI prototypes to align people ("people are visual… need to see what's involved"). - Nina says PMs already use v0 for interactive mockups in user research. - Raj calls AI-drafted PRDs "could work" and says engineering does much of the scoping itself. He also flags too many meetings eating senior IC and management time. **Suggested content:** quick throwaway prototypes for alignment, AI-assisted drafting of lighter-weight tech specs split by purpose, and using AI to summarize and share context so senior people can skip some meetings. Aim this at tech leads and PMs. It helps, but it isn't what's holding back engineering output. --- ## Audience: design for three groups The interviews point to a spread of skill levels and attitudes, so one-size training will bore some people and lose others: | Segment | Example | Needs | |---|---|---| | Power users | Ben | Advanced agents and subagents, prompt optimization, writing the shared rules. **Use them as co-teachers.** | | Everyday users | Nina (self-described "basic") | Context techniques, incremental prompting, first background-agent tasks. | | Skeptical or experienced | The group Raj describes | Evidence on real Northcove code, the risk-tier framework, compliance clarity. Don't lead with hype. | Suggested format: a shared core (Themes 1–3, about half a day to a day, hands-on in the real monolith using Cursor since it's the enterprise standard) plus optional deep dives (Themes 4–5), followed by office hours and a 30-day follow-up run through the existing Slack channels. ## Tangible outputs the training should produce - A version-controlled **Northcove rules and commands pack** (per team or area), with a named owner so it doesn't go stale like the docs have. - A written **risk-tier policy** for AI-assisted work, agreed with Legal and Compliance. - A **starter library of agent task recipes** (test coverage, flag removal, bug reproduction). - (Optional track) An **eval harness template** for LLM features. ## How to measure it Share of PRs with meaningful agent contribution; background-agent tasks per engineer per week; adoption of the rules pack (repos with rules checked in); test coverage in the areas agents touch; self-reported confidence before and after, split by segment; and whether AI-related incidents, especially data-handling ones, stay at zero. --- ## Caveats and assumptions - **Small, uneven sample.** 3 of about 100 engineers. Ben is in Business and Nina's group isn't stated. Consumer and Platform may have different needs, so check them with a short survey or 2–3 more interviews before finalizing. - **Raj's role is assumed** to be a manager or org-level leader, since his notes describe the whole organization rather than his own work. - **A conflicting data point:** Raj says engineers rarely hand-write more than 20 lines, while Ben (a power user) says he rarely uses AI for production features. My read is that AI is used everywhere for autocomplete and small edits but rarely for substantial production changes. That gap is exactly what Themes 1–3 target. Worth confirming. - **Tooling:** I assumed Cursor as the main teaching platform (enterprise license, universal adoption). Claude Code has only been experimented with, so treat it as optional for power users. - **Non-engineering adoption** (Cursor-in-Slack for PMs, BI hookup) came up but is out of scope for *engineering* training. Flag it as a follow-on opportunity.