# AI Engineering Training for Northcove — Synthesis & Recommendations

*Based on meeting notes with Nina Park (Full-Stack Engineer), Ben Walsh (Senior Engineer, Business), and Raj (Engineering Org overview). Assumption: The audience is Northcove's ~100 engineers. Duration/format not specified, so this recommends a core curriculum + advanced track that can be delivered as a half-day intensive or a 3-4 session series.*

---

## 1. Executive Summary: What to Focus On

Northcove is **not starting from zero** — and the training shouldn't act like it is.

Everyone is on Cursor Enterprise. Tab-complete is ubiquitous. The team is, in Raj's words, now "editors rather than authors — rarely write more than 20 lines by hand." But adoption is **uneven and shallow**:

* Ben is a power user (never hand-writes a rough draft, experimenting with background agents/subagents, wants DSPy). Nina self-describes as a "basic user" (Cursor chat for scaffolding, tests, debugging).
* Most of the org is still **"AI-second, not AI-first"** (Ben).
* Agents are being experimented with, but are **not yet core workflow** for anyone.
* Production-feature coding is the *least* trusted use case, due to Northcove-specific coupling, legacy antipatterns, and regulated data.

**The training gap is not "how to prompt ChatGPT." It's Northcove-specific context, verification, and evals.**

Prioritized focus for the training, in order:

| # | Focus Area | Why It's #1 Driver |
| :--- | :--- | :--- |
| 1 | **Context Engineering** | All 3 cite this unsolved; it's Ben's "Context is king" principle and Nina/Raj's #1 failure mode |
| 2 | **Verification, Testing & Guardrails** | Correctness-critical business + plausible-but-wrong AI code + weak E2E coverage = biggest risk |
| 3 | **Evals for LLM Features** | Only live LLM product (doc interpretation) has manual, labor-intensive evals; this is Northcove's differentiation skill |
| 4 | **Background Agents & Incremental Delegation** | Highest curiosity (Nina + Ben), lowest current competence, high payoff if done safely |
| 5 | **AI-First Planning & Prototyping** | Already emerging organically (v0 mockups, Ben's prototypes) but not standardized; fixes bloated tech specs |
| 6 | **Northcove Rules, Prompts & Tooling (MCP vs CLI)** | No shared rules/commands library today — everyone is re-inventing and hitting legacy patterns |
| 7 | **Compliance & Data Boundaries** | Cross-cutting constraint. Regulated data, prior 3rd-party incident, litigation caution — must be taught *inside* every module, not as an afterthought |

**Do NOT center the training on:** generic prompt-engineering tricks, writing code from scratch with AI, or autonomous building of non-trivial features / DB migrations / user research. All three engineers are skeptical or say the tools aren't ready there, and centering on it will lose the experienced (and already AI-skeptical) senior cohort.

---

## 2. Company Context the Training Must Be Built Around

These facts from the notes constrain what will work:

* **Scale & culture:** 100 engineers (Consumer / Business / Platform silos), doubled YoY, typically 4–10 yrs experience. Strong autonomy culture; PRDs are thin, engineers do a lot of scoping. More experienced engineers are *more* suspicious of AI.
* **Stack:** Python backend, TypeScript frontend, monolithic app, standard libraries only. CI 20–30 min, deploy 40 min–2 hrs, continuous deployment. Projects are mostly single-engineer, 1 week (bets) to 5–6 weeks (infra).
* **The one production LLM feature:** Document interpretation (Nina, v2) — LLM scans uploaded documents with varying layouts/terminology, using an Anthropic model under organizational data-isolation contracts. This is the best internal case study you have. Use it.
* **Business risk profile:** Business group does correctness-critical, multi-party money calculations (estimates, invoicing, Stripe, loss per transaction, recoups/refunds). "If you turn one knob, something else falls down elsewhere." This is not a "move fast and let AI ship it" org.
* **Knowledge problem:** Docs are stale and scattered across Google Docs, Confluence, and Slack. There is *no structured way to feed docs to Cursor*. This is an engineering-systems problem, not just a training problem.
* **Compliance overhang:** Prior third-party data-sharing incident requiring rework, plus litigation concerns. Legal/compliance factors into *every* decision. Any training that doesn't give engineers a crisp "what can / cannot go into a model" rule will either be ignored or will create fear.

---

## 3. Thematic Synthesis — What the Three Engineers Converge On

### Theme 1: Context is the bottleneck, not prompting cleverness

* **Ben:** "Context is king — don't just say 'do the thing,' tag relevant files upfront." Quality output requires standardized Northcove-specific prompts/guidelines. Org buy-in needed for rules/commands/reusable prompts.
* **Nina:** AI over-engineers on complex prompts and "cherry-picks wrong context from legacy code patterns." Works far better with *concrete examples from the codebase* than general prompts. Has only a basic system prompt (Python/framework specs); can't yet encode detailed standards.
* **Raj:** AI is "not good at Northcove-specific implementations." Needs "very prescriptive custom rules." Knowledge in Confluence/Slack/docs goes stale and is hard to find.

**Implication:** The single highest-leverage module is teaching engineers to *curate* context: which files to tag, how to write Cursor Rules / `CLAUDE.md`-style project instructions for Python/TS conventions, how to give one golden example instead of a paragraph of instructions, and how to recognize when the model has latched onto a legacy antipattern and intervene.

### Theme 2: AI output looks right and is subtly wrong — verification is underbuilt

* **Nina:** "AI code looks plausible but may introduce subtle issues." Lack of robust E2E/integration testing. Risk of accepting "good enough" output. Need better monitoring/validation.
* **Ben:** Deliberately does *not* use AI much for production features — can't supply all context, hidden side effects and coupling from fast startup development. Uses it heavily for throwaway sketches, planning, debugging, one-off scripts instead.
* **Raj:** Confidence is highest for dev-assistance codegen *with human review, especially for database/regulated data*. Skeptical for significant non-trivial features and DB migrations.

**Implication:** Training must teach a verification discipline, not just generation. Diff-review patterns, writing the test *first* and having AI pass it, using AI to write tests for existing suites (already Nina's workflow — formalize it), and a clear tiering: throwaway/internal (directionally correct is fine, per Ben) vs. production correctness-critical (adversarial review mandatory).

### Theme 3: Evals are manual, painful, and where the real LLM-product skill lives

Only Nina is building an LLM feature, but her pain is the clearest signal of a capability Northcove lacks org-wide:

* Manually labeled 100–250 documents as ground truth. Iterative prompt tweaking is *more* labor-intensive than traditional ML training. Small prompt changes cause non-deterministic swings. Only recently shifting to larger-scale proxy metrics.

**Implication:** If Northcove ships more LLM features (and document intake sits in Ben's group too), "prompt and pray" will not scale. Engineers need eval literacy: building golden datasets, choosing proxy vs. human labels, regression-testing prompts, and (advanced) automated prompt optimization. Ben already knows about DSPy but has had no time to implement it — this is the advanced-track hook.

### Theme 4: Everyone wants background agents; nobody has a safe pattern yet

* **Nina:** Current workflow is "very back-and-forth, time-consuming." Wants long-running background processes, better prompting for *incremental* changes, delegating larger features to independent agents.
* **Ben:** Starting to use agents for test coverage, feature-flag removal, debugging. Using subagents, but "not core to workflow yet." Active Slack channels (`cursor-nerds`, `background-agents`) show grassroots demand.
* **Raj:** "Agents less adopted, some early adopters experimenting."

**Implication:** Teach agents as *scoped, reviewable delegation* — small briefs, checkpoints, incremental diffs — not "give it a feature and walk away." Anchor on Ben's proven use cases (test coverage, flag removal, debugging) and Nina's pain (incremental changes), and explicitly pair this module with Theme 2 guardrails.

### Theme 5: AI is already reshaping planning — formalize it

* **Ben:** Quarterly tech specs are "too much" — one doc trying to do strategy, deadlines, resourcing, sequencing, *and* technical alignment. Hard to write, hard to review. He uses AI-generated prototypes for alignment: "People are visual… prototypes just need to be directionally correct, don't need to compile. Cheap to throw away now."
* **Nina:** PM uses Vercel v0 mockups for interactive user testing, enabling non-technical iteration without Figma or engineer-built prototypes.
* **Raj:** PRD development "could work" with AI but isn't used due to product-org workflows; most projects start from thin PRDs anyway.

**Implication:** A short, practical module on AI-first planning will land well because it solves a *felt* process pain (bloated specs, slow alignment), not just a coding task. Teach: prototype-to-align, disposable sketches before specs, splitting the mega-spec into smaller AI-drafted artifacts, and Ben's "directionally correct is the name of the game" for internal tools.

### Theme 6: Tooling beyond autocomplete is fragmented (MCP, rules, sharing)

* **Ben:** Not effectively using MCP beyond DB connections; tried Confluence/Jira/GitHub MCP but hit performance issues; *prefers CLI tools (in the training set) over configuring MCP servers*. Aware of DSPy, no time to implement.
* **Raj:** Early Cursor adopters, experimented with Claude Code. Chat-for-planning-then-building (Cursor-team recommended) and code search are established.
* **All:** Sharing happens ad-hoc in Slack (`cursor-nerds`, `background-agents`, eng channels). Raj flags siloed knowledge management as a "major gap."

**Implication:** Don't teach MCP configuration as a centerpiece — Ben's experience suggests CLI-first will get more traction. Instead, teach *when* to reach for MCP vs CLI vs native search, and spend the time on building a shared, version-controlled library of Northcove Rules, commands, and reusable prompts that the org actually maintains.

### Theme 7: Compliance is the constraint that can veto everything else

* **Raj:** Concern about regulated-data exposure; prior incident; litigation caution shapes decisions.
* **Nina:** Document feature runs on Anthropic with *organizational data isolation contracts* — engineers need to understand what that does and doesn't cover.
* **Ben:** DB/regulated work is where human review is non-negotiable.

**Implication:** Embed a 15-minute "data boundaries" segment up front (what data classes exist at Northcove, what's approved for Cursor/Anthropic under current contracts, what never gets pasted) and repeat the relevant rule inside each hands-on lab. A standalone legal lecture at the end will be tuned out.

---

## 4. Recommended Training Design

### Audience segmentation

| Segment | Who | What they need |
| :--- | :--- | :--- |
| **Core (all engineers)** | ~100 engineers, mixed AI confidence | Modules 1–3, 5, 7 below |
| **Power-user / champion track** | Ben and peers, Slack-channel regulars, early agent experimenters | Modules 4, 6 + DSPy/eval-optimization deep dive |
| **Optional extension** | PMs, BI, non-eng Cursor users (already asking via Slack integration) | Lightweight version of Module 5 (prototyping) only — don't dilute the engineering core for this |

Explicitly plan for the skeptical senior cohort Raj flagged: lead with limitations and review discipline, use their correctness-critical examples, and recruit Ben-type power users as co-instructors/demo-ers rather than external evangelists.

### Core curriculum (priority order)

**Module 1 — Context Engineering at Northcove (hands-on, Python/TS monolith)**
Tagging relevant files upfront; writing and maintaining Cursor Rules for Northcove Python/TS conventions; example-driven prompting (one golden file > long instructions); spotting legacy-antipattern contamination and correcting it; building a team context pack where docs are stale. *Lab: take a real Business-group-style coupled change, do it once with a vague prompt (watch it over-engineer), once with curated context + example.*

**Module 2 — Verification & Guardrails for AI-Generated Code**
Review-tier framework (throwaway / internal tool / production / correctness-critical & regulated). Test-first AI workflows; using AI to raise test coverage safely; diff-review checklists for subtle issues (side effects, coupling, DB access); when *not* to use AI (non-trivial features, migrations, regulated data — per Raj's confidence ladder). *Lab: seed a plausible-but-buggy AI diff in a money-calculation path; teams find the subtle bug and write the guardrail test.*

**Module 3 — Evals for LLM Features (using document interpretation as the case study)**
Golden datasets (how Nina's 100–250 doc set was built, and its limits); human labels vs proxy metrics and when to switch; prompt regression testing under non-determinism; error analysis on varying layouts/terminology. *Lab: given a small labeled doc set, tweak a prompt, measure against the set, observe the non-deterministic swing, add a regression check.* Advanced follow-on: DSPy-style automated prompt optimization (Ben's stated interest).

**Module 4 — Background Agents & Incremental Delegation [Champion track, then roll out]**
Scoping an agent brief; incremental prompting for small diffs vs big-bang feature requests; checkpoint/review cadence; proven starter tasks (test coverage, feature-flag removal, debugging, one-off scripts). Failure modes and abort criteria. *Lab: delegate a flag-removal or test-coverage task to a background agent with a checkpoint plan.*

**Module 5 — AI-First Planning & Prototyping**
Disposable prototypes for alignment (v0, Cursor sketches); replacing the mega tech-spec with smaller artifacts; Ben's prototype-before-spec workflow and Raj's thin-PRD reality; "directionally correct" standard for internal tools. *Exercise: turn a bloated spec excerpt into a throwaway prototype + 1-page alignment doc in 30 minutes.*

**Module 6 — Shared Northcove Toolkit [Champion track + working session, not a lecture]**
Building the org's version-controlled Rules/commands/prompt library; MCP vs CLI decision guide (CLI-first per Ben's experience, MCP where it earns its keep, e.g. DB); how to contribute patterns back via existing Slack channels into the library. *Output of the session is the actual v1 library — training that ships an asset.*

**Module 7 — Data Boundaries & Compliance (15 min up front + reinforced in every lab)**
Data classification at Northcove; what the Anthropic isolation contract covers; regulated-data handling and human-review requirements; lessons from the prior third-party incident (at whatever level of detail Legal approves). *Deliverable: a one-page "safe / review / never" card engineers keep open while coding.*

### What to cut or defer

* Generic "10 prompt hacks" content — this audience is past it and seniors will disengage.
* Autonomous end-to-end feature building and DB-migration automation — explicitly marked "not ready" by Raj; teaching it now undermines credibility.
* Deep MCP server configuration — low traction, performance complaints; cover as a decision guide only.
* AI for user research — Raj notes only ~3 research contacts; insufficient volume.

---

## 5. Sequencing & Format Recommendations

1. **Start with champions.** Run Module 4/6 with Ben and the `background-agents`/`cursor-nerds` regulars first. They become demo-ers and library maintainers for the core rollout, and their real Northcove examples replace generic ones.
2. **Core delivery:** 3 sessions × ~90 min over 2–3 weeks (Modules 1+7, 2, 3+5 interleaved), each with a lab on the real Python/TS monolith — not toy examples. Single-engineer project norms mean labs should be solo/pair-sized and finishable.
3. **Ship assets, not just knowledge:** Cursor Rules v1, prompt/command library, eval template from the doc-interpretation case, and the compliance one-pager. The stale-docs problem means unmaintained training material will rot — assign owners (champions) and a review cadence.
4. **Measure it:** Baseline and follow up on (a) % of PRs using shared Rules, (b) test coverage on AI-touched code, (c) eval coverage on LLM features (golden set size, regression runs per prompt change), (d) background-agent tasks completed with review pass. Avoid vanity metrics like "AI lines generated" — Raj's "editors not authors" shift makes line counts meaningless.

## 6. Risks If You Train the Wrong Things

* **Training only generation, not verification**, in a correctness-critical money system with weak E2E coverage, increases the "plausible but subtly wrong" incident rate Nina flagged.
* **Ignoring the context/docs problem** means every technique taught degrades against stale Confluence/Slack knowledge. Pair the training with even a minimal effort to curate a Cursor-feedable context source for the highest-traffic domains.
* **One-size-fits-all delivery** will bore power users (Ben) and overwhelm basic users (Nina) simultaneously. The champion/core split above is the fix.
* **Compliance as an afterthought** risks either a regulated-data incident or blanket fear that kills adoption. Give engineers crisp rules early.

---

## 7. One-Paragraph Brief (for stakeholders)

Northcove's engineers already generate most of their code with AI but lack shared, Northcove-specific practices for the things that actually determine quality and risk: curating context in a coupled Python/TypeScript monolith with stale docs, verifying plausible-but-subtly-wrong output in correctness-critical money flows, and evaluating their one production LLM feature (document interpretation) beyond manual prompt-tweaking. The training should therefore focus, in order, on context engineering, verification/guardrails, and LLM evals, with background-agent delegation and AI-first prototyping as high-interest follow-ons, a shared Rules/prompt library built *during* the training, and data-boundary rules embedded throughout given regulated-data and litigation sensitivity. Generic prompt training, autonomous feature building, and DB-migration automation should be explicitly deferred — the org's own engineers rate them not-ready, and leading with them would lose the experienced cohort whose buy-in the rollout depends on.

