04 thematic

AI engineering training focus for Northcove

Executive recommendation

Northcove should train engineers to become effective AI-assisted system engineers, not merely better prompt writers. The organization already has broad access to Cursor and substantial comfort with autocomplete, code search, planning, throwaway prototypes, tests, scripts, and debugging. The training should therefore concentrate on the harder transition:

from “AI can produce plausible code” to “AI can make useful, context-aware changes that are verified, compliant, and maintainable.”

The highest-value curriculum has five connected capabilities:

1. Context engineering: giving AI the right Northcove-specific files, rules, examples, constraints, and definitions.
2. Safe delegation: breaking work into plans, bounded tasks, background agents, and subagents with explicit stopping points.
3. Verification and evaluation: testing AI output—including LLM features—with the rigor required for business logic and regulated data.
4. AI-native engineering workflows: using AI across discovery, planning, debugging, documentation, operations, and code—not only during implementation.
5. Reusable organizational infrastructure: turning scattered knowledge and individual experimentation into shared rules, prompts, commands, tools, and playbooks.

What the interviews consistently say

Strengths to build on

• Cursor is already broadly adopted, especially autocomplete, code search, chat-based planning, and basic code generation.
• Engineers are comfortable using AI for prototypes, scripts, test generation, debugging, explanations, and repetitive cleanup.
• There is interest in agents, background work, subagents, prompt optimization, and AI-generated prototypes.
• AI is spreading beyond engineering, so a strong engineering foundation could benefit PM, BI, and other functions.

The common constraints

• AI does not reliably understand Northcove’s implementation patterns, hidden coupling, legacy behavior, or domain rules.
• Company knowledge is fragmented across Confluence, Google Docs, Slack, Jira, and code; much of it is stale or difficult to retrieve.
• Production changes are riskier than prototypes, particularly database migrations, business-critical calculations, integrations, and regulated-data workflows.
• AI output often looks correct while containing subtle defects. End-to-end, integration, monitoring, and evaluation practices are not yet strong enough to catch all of them.
• Engineers use different, largely informal workflows. Useful techniques are shared socially but are not yet standardized.

Recommended curriculum, in priority order

1. Context engineering for Northcove codebases

This should be the first module because all three interviews identify context as the main quality bottleneck.

Teach engineers to:

• start with repository reconnaissance and dependency mapping before asking for implementation;
• identify the authoritative files, domain concepts, invariants, interfaces, and examples an agent needs;
• use small, relevant context sets instead of dumping a whole repository into a prompt;
• provide “golden examples” of Northcove patterns and explicitly call out anti-patterns;
• create and maintain scoped project rules for Python, TypeScript, the monolith, testing, data access, observability, and review expectations;
• ask the model to state its assumptions and list the files and behavior it believes are affected;
• distinguish current source-of-truth documentation from historical or untrusted material.

The practical output should be a repeatable Northcove context packet or checklist for any non-trivial task. Training should use real examples from each silo—Consumer, Business, and Platform—because generic examples will not address the central problem.

2. Plan-first, bounded agent workflows

Northcove is ready to move beyond conversational back-and-forth, but delegation needs structure. Teach a workflow such as:

1. ask the agent to inspect and produce a plan;
2. review assumptions, scope, risks, and proposed tests;
3. delegate one bounded slice at a time;
4. require the agent to run specified checks and report changed files and unresolved questions;
5. review the diff and evidence before merging;
6. stop or roll back when the task crosses a defined risk boundary.

Include hands-on practice with background agents and subagents for low-risk work first: test coverage, feature-flag removal, debugging, documentation, dependency or usage analysis, and one-off internal tools. Then show how to coordinate multiple agents without losing a single owner, clear acceptance criteria, or traceability.

The emphasis should be on task decomposition and control surfaces—not on letting an agent independently build a large production feature end to end.

3. Verification, testing, and evaluation of AI output

This is the most important risk-control module. It should cover both ordinary AI-generated code and LLM-powered product features.

For code, teach:

• test-first or test-before-acceptance prompting;
• unit, integration, end-to-end, regression, and contract tests, with particular attention to hidden side effects;
• static checks, type checks, database safety checks, and migration review;
• adversarial review prompts that look for authorization, data leakage, race conditions, incorrect edge cases, and failure handling;
• requiring executable evidence rather than accepting an agent’s claim that tests passed;
• staged rollout, feature flags, observability, and rollback criteria.

For LLM features, teach:

• how to construct and maintain labeled evaluation sets;
• deterministic and non-deterministic evaluation strategies;
• proxy metrics versus human-judged quality;
• slice-based analysis by document type, customer/source, terminology, and failure mode;
• prompt/model regression testing;
• error taxonomies and monitoring after release.

Nina’s document-interpretation work is an excellent case study: it makes the labor of ground-truth labeling, prompt sensitivity, and evaluation design concrete.

4. Safe AI engineering under compliance and data constraints

This must be a required track, not an optional security lecture. Engineers need clear operating rules for using AI with Northcove data and systems.

Cover:

• what data may, may not, or may only conditionally be sent to an AI tool;
• regulated data, customer documents, secrets, credentials, production database access, and third-party integrations;
• data isolation, retention, logging, model/provider boundaries, and approved enterprise configurations;
• safe use of anonymized or synthetic fixtures;
• review requirements for database changes, financial/business calculations, and customer-facing behavior;
• prompt injection and untrusted-document risks in document interpretation and retrieval workflows;
• how to document model/vendor decisions and escalate uncertainty.

The training should provide a short decision tree and concrete examples, so engineers can make a safe decision during a fast one-week experiment rather than relying on vague caution.

5. AI-first workflow across the engineering lifecycle

AI should be positioned as a collaborator throughout the lifecycle:

• discovery: summarize customer and operational evidence, identify unknowns, and draft questions;
• planning: turn goals and metrics into acceptance criteria, risks, slices, and technical options;
• prototyping: generate directionally correct artifacts for alignment and rapid disposal;
• implementation: scaffold, explain, search, refactor, and extend code;
• debugging: reproduce, narrow hypotheses, inspect logs, and propose experiments;
• operations: analyze incidents, CI/deploy bottlenecks, cost, and recurring support work;
• knowledge work: draft documentation, PR descriptions, decision records, Jira updates, and handoffs.

This should connect to Ben’s point that AI’s value is broader than production feature coding. It also addresses Raj’s concern about meeting load by making asynchronous summaries, decision capture, and searchable context part of the workflow.

6. Shared Northcove AI tooling and knowledge practices

Training alone will not solve the context problem. Reserve time for engineers to build and adopt reusable organizational assets:

• a version-controlled rules repository for Cursor/Claude and approved command patterns;
• task templates for planning, implementation, debugging, review, migration, and incident response;
• a small library of Northcove-specific examples and anti-patterns;
• standard commands or scripts for repository search, test execution, dependency inspection, and evidence collection;
• a documented approach for linking or ingesting authoritative Confluence/Jira/GitHub material;
• a lightweight mechanism to mark stale documentation and identify owners;
• approved integrations or CLI alternatives where MCP performance is inadequate.

The goal is not to configure every possible MCP server. Ben’s experience suggests that fast, reliable CLI tools may be more useful initially. Teach the selection criteria—latency, permissions, auditability, freshness, and failure behavior—then standardize only the integrations that earn their place.

Suggested delivery format

Use a two-layer program:

Core workshop for all engineers (one day or two half-days):

• Northcove AI safety and data rules;
• context engineering;
• plan-first Cursor/agent workflow;
• verification and review;
• a hands-on exercise using an existing Northcove repository.

Role-based labs and office hours (over 4–6 weeks):

• Business: correctness-critical calculations, financial flows, Stripe/integration behavior, and migration safety;
• Consumer: product experiments, prototypes, frontend/backend changes, and end-to-end validation;
• Platform: CI/CD, deployment time, observability, AWS cost, tooling, and agent infrastructure;
• LLM feature builders: eval sets, prompt/model regression, document safety, and production monitoring;
• senior engineers and tech leads: delegation design, review standards, and team-level adoption.

Each participant should apply the methods to a real, bounded task and publish the resulting context packet, plan, diff, tests, and evaluation evidence. This creates reusable examples for the next cohort.

What not to make the center of training

• Basic prompting or generic “how to use ChatGPT” instruction; adoption has already moved past that.
• A survey of every new AI tool, model, or MCP server; tool churn will outpace the learning value.
• Fully autonomous implementation of large, high-risk production features as the default aspiration.
• AI-generated user research conclusions from tiny samples; Raj explicitly flags this as not ready.
• Prompt optimization as a standalone advanced topic before the organization has stable evals and reliable data. DSPy or similar methods become more useful after those foundations exist.

Measures of success

Track outcomes, not prompt volume. A practical initial scorecard could include:

• percentage of teams using shared Northcove rules and task templates;
• percentage of AI-assisted PRs with documented context, tests, and verification evidence;
• change-failure rate, escaped defects, rollback rate, and review rework for AI-assisted changes;
• time from task start to a reviewed PR, separated by low- and high-risk work;
• test and integration coverage for areas where agents are used;
• LLM feature quality by evaluation slice and regression rate after prompt/model changes;
• number of compliance or data-handling exceptions;
• engineer confidence in knowing when AI is appropriate, what context to supply, and how to verify the result;
• reduction in repeated questions and meeting time attributable to better captured knowledge.

Do not optimize for “lines of code generated” or raw Cursor usage. Those metrics can rise while quality and safety fall.

Bottom line

The training should create a common Northcove method for context → plan → bounded delegation → verification → learning. Start with the organization’s existing AI fluency, then invest most heavily in context quality, risk-aware delegation, evaluation, and shared engineering infrastructure. That combination is most likely to unlock faster experimentation without increasing the subtle correctness, compliance, and maintenance failures the interviews repeatedly surface.

Assumption

This synthesis is based on notes from three engineers and should be treated as a strong directional read, not a full organizational diagnosis. It assumes the training audience is primarily engineers across Consumer, Business, and Platform, with some future enablement for PM, BI, and other functions.