# What Northcove should focus on in AI engineering training

Focus the training on **making AI-assisted work reliable in Northcove’s Python/TypeScript monolith: supplying context, scoping changes, and verifying results.** Engineers already have access to Cursor and use it for useful work. The next gain is helping them complete more consequential tasks with less back-and-forth and a clear basis for trusting the result.

This synthesis uses the three interviews in [context.md](context.md). Assumption: the primary audience is software engineers, with mixed levels of AI experience; training duration has not been set. The proposed format below is a starting point. Three interviews provide directional evidence, not an organization-wide assessment.

**What the interviews collectively tell us**

- **Company-specific context is the clearest shared bottleneck.** Nina sees AI copy inappropriate legacy patterns; Ben cannot convey hidden coupling and side effects; Raj says Northcove-specific implementations require prescriptive rules. Generic prompting advice will have limited value without exercises grounded in the actual codebase.
- **Confidence in correctness limits delegation.** Nina worries about plausible code and weak integration coverage. Ben uses AI extensively but avoids much production feature coding in correctness-critical business logic. Raj is skeptical of significant features, migrations, and regulated-data work. Training should make verification a repeatable part of the workflow.
- **Adoption is uneven, and usage is different from trust.** Raj describes engineers as editors who rarely write much code manually; Ben describes much of engineering as “AI-second”; Nina considers herself a basic user. These accounts may reflect different teams and tasks. Avoid designing for either universal beginners or universal power users.
- **Some constraints require organizational work.** Stale documentation, hidden coupling, testing gaps, and infrastructure problems will persist after a workshop. Training can establish useful habits and reusable artifacts, but owners must maintain them.

**Recommended priorities, in order**

| Priority | What to teach | Evidence and practical exercise |
|---|---|---|
| **1. Build a useful context package** | Select relevant files, a current implementation example, business rules, dependencies, and explicit constraints. Ask AI to identify missing context before proposing changes. Create small, scoped repository instructions with clear ownership. | All three report context failures. Have participants prepare a task brief for a real Python/TypeScript change, identify a misleading legacy example, and compare the resulting plans. |
| **2. Plan, implement, and verify a bounded change** | Define acceptance criteria, inspect affected paths, request incremental changes, review the diff, and choose checks based on what could break. Use independent expected outcomes to evaluate generated tests. | Nina’s validation concerns, Ben’s calculation domain, and Raj’s production skepticism. Use a sanitized business-logic example with known edge cases; require participants to catch a plausible but incorrect implementation. |
| **3. Delegate work with explicit boundaries** | Choose suitable tasks, specify permitted scope and completion evidence, define when the agent must stop or ask, and review the handoff. Start with one agent before introducing subagents. | Nina wants less back-and-forth; Ben has begun background tasks; Raj sees limited agent adoption. Practice on a bounded test-coverage task or internal tool, with a reviewer checking both correctness and scope. |
| **4. Use AI for investigation and technical alignment** | Map a code path, reproduce a bug, draft a concise implementation brief, and build a disposable prototype to expose assumptions. Separate decisions, open questions, and implementation details. | Ben finds planning/debugging useful and tech specs overloaded; Raj describes substantial engineer-led scoping and meeting load. Turn an ambiguous request into a short brief and prototype for review. |
| **5. Evaluate LLM-powered product features — specialist track** | Maintain labeled evaluation cases, separate development from held-out evaluation, analyze errors by document type, compare prompt/model versions, and quantify variability. Treat proxy metrics as supplementary evidence. | Nina already has 100–250 labeled documents and sees unstable results; Ben is interested in prompt optimization. Use a sanitized document-interpretation exercise to decide whether a proposed prompt change actually improves performance. |

**How to run it**

Use two core workshops, followed by optional clinics. Most time should go to hands-on work and review.

1. **Core workshop 1: context and trustworthy changes.** Briefly establish a shared workflow, then work through priorities 1 and 2 on a representative repository task. Pair engineers across experience levels. Experienced users can tackle hidden dependencies and stronger verification while newer users practice file selection and incremental edits.
2. **Core workshop 2: delegation and planning.** Work through priorities 3 and 4. Each participant delegates a bounded task and reviews another participant’s output against its acceptance criteria. Include a short brief or prototype exercise that helps clarify an engineering decision.
3. **Optional clinics:** background agents/subagents for engineers ready to extend delegation; evaluation and prompt optimization for engineers building LLM features. Establish a credible evaluation baseline before optimizing prompts.

Ask Northcove to supply a sanitized repository example, a known-good implementation pattern, and its approved tool/data-handling guidance before delivery. Integrate that guidance into exercises involving customer data, database access, and external tools. Do not infer company policy from these interviews.

Ben’s tolerance for directionally correct prototypes is useful for exploration. For production changes, particularly calculations and regulated-data workflows, exercises must require explicit correctness evidence. Account for the reported slow CI/deploy cycle by teaching focused local checks followed by the required integration and CI checks.

**What participants should leave with**

- A reusable task brief covering goal, relevant context, constraints, acceptance criteria, and verification.
- A small set of reviewed Northcove-specific repository instructions, with owners and a maintenance process.
- A delegation template covering scope, stop conditions, and required handoff evidence.
- One completed, reviewed example showing how to move from investigation to a validated change.

Consolidate these artifacts in one discoverable home. Northcove should separately assign ownership for keeping documentation and approved examples current, and for addressing the testing gaps the exercises expose.

**What to de-emphasize**

Keep generic AI introductions brief. Avoid making broad tool tours, MCP configuration, or autonomous large-feature demonstrations central to the course: these do not address the strongest shared blockers. Defer company-wide knowledge search and BI integration to separately scoped initiatives. PRD generation and automated user research also have weaker support in these interviews than engineering context, verification, and delegation.

**How to judge whether it worked**

Capture a baseline on comparable tasks, then review outcomes after two to four weeks: time to an accepted change including review and rework; correction cycles; defects or regressions; and whether reusable instructions remain useful and current. Inspect a small sample of completed tasks for acceptance criteria and verification evidence. For the LLM-feature track, assess held-out task accuracy and important error categories, with proxy metrics alongside them.

The success criterion is that engineers can complete an appropriate task with less supervision and demonstrate why the result is correct. More generated code or more agent usage alone would not establish that.