Thematic
gemini-4-argonrun 4 Oct 202676,483 tokens
Instructions
Instructions — what the model was asked
help me synthesize what to focus on in our AI engineering training for this company based on meeting notes with three engineers
help me synthesize what to focus on in our AI engineering training for this company based on meeting notes with three engineers
Northcove AI Engineering Training: Synthesis & Curriculum Strategy
Synthesized from discovery interviews with Nina Park (Full-Stack Engineer, AI Feature Builder), Ben Walsh (Senior SWE, Business Group / AI Power User), and Raj (Engineering Leadership / Org Overview).
1. Executive Synthesis: Where Northcove Stands Today
Northcove (~100 engineers across Consumer, Business, and Platform; Python/TypeScript monolith) has achieved high baseline adoption of AI tooling—enterprise Cursor licenses are rolled out, tab-autocomplete and basic chat are ubiquitous, engineers act as "editors rather than authors" (rarely writing >20 lines manually), and the Cursor Slack integration is wildly popular even among non-engineers.
However, the organization is stuck at a local maximum ("AI-Second" mode). Across the three interviews, four systemic tensions explain why AI works well for greenfield scaffolding and throwaway scripts, but breaks down on real Northcove production work:
- The Monolith Context Trap: Fast startup growth left Northcove's Python/TS monolith with tight coupling, hidden side effects ("turn one knob, something else falls down" — Ben), and legacy antipatterns. Without structured context or prescriptive rules, Cursor cherry-picks legacy patterns and over-engineers solutions (Nina), fails on Northcove-specific implementations and database migrations (Raj), and forces even power users to avoid AI for core production features (Ben).
- The Verification & Guardrail Gap: Engineers generate code faster than ever, yet Nina highlights a lack of robust integration/E2E testing and the danger of accepting "plausible-looking" AI code with subtle bugs. In a codebase with correctness-critical multi-party financial logic (Ben), slow feedback loops (20–30 min CI, 40–120 min deploys — Raj), and high sensitivity around regulated data and a prior third-party data-sharing incident (Raj), unverified AI code creates unacceptable risk—fueling skepticism among experienced engineers (4–10 YoE).
- Disconnected Pockets of Excellence: Solutions to one engineer's pain points already exist in another corner of the org, trapped in Slack channels (
#cursor-nerds,#background-agents):- Nina is exhausted by back-and-forth chat and wants background agents for test writing and incremental features; Ben is already running background agents for test coverage and feature-flag cleanup.
- Nina is manually tweaking prompts and hand-labeling 100–250 documents for the Anthropic-powered document interpretation system; Ben already knows automated prompt optimization (DSPy) is needed across projects.
- Nina and Raj lament stale docs across Confluence, Google Docs, and Slack; Ben has already tested MCP servers (finding them too slow/flaky for Jira/Confluence/GitHub) and proven that CLI tools + upfront file tagging work far better.
- Process & Meeting Bloat: Senior ICs and managers are drowning in synchronous meetings (Raj) and bloated quarterly tech specs that try to solve strategy, resourcing, sequencing, and technical design in a single document (Ben), while PRDs arrive under-scoped (Raj).
2. Core Training Focus Areas (What to Teach)
Because Northcove's engineers are experienced (4–10 years) and rightly skeptical of generic AI hype, training must not be a generic "Intro to Prompting" workshop. It must be a pragmatic, monolith-grounded program focused on five pillars:
Pillar 1: Context Engineering & Repo-Level Rules for the Monolith
Addresses: AI cherry-picking legacy antipatterns (Nina), hidden coupling in production code (Ben), poor company-specific codegen & stale Confluence/Slack docs (Raj).
- "Context is King" Workflow: Train engineers to stop issuing open-ended prompts ("build X") and instead anchor every session by explicitly tagging canonical reference files, target interfaces, and test files upfront (Ben's core principle + Nina's finding that concrete codebase examples prevent over-engineering).
- Standardized
.cursor/rules& Golden Examples: Shift Northcove's tribal knowledge out of stale Confluence/Google Docs and into version-controlled, directory-scoped rules (e.g.,.cursor/rules/CLAUDE.md). Teach teams how to write prescriptive rules that:- Point AI to canonical "Golden Pattern" files in Python and TypeScript.
- Explicitly blacklist deprecated legacy patterns and flag high-coupling modules ("Do not modify X without updating Y").
- Codify Northcove's database migration conventions and infrastructure constraints (Raj).
- Pragmatic Tooling — CLI Over Fragile MCPs: Share Ben's hard-won lesson across the org: avoid spending hours debugging slow third-party MCP servers for GitHub, Jira, or Confluence. Instead, teach engineers to equip agents with standard CLI tools (
gh,pytest, custom Python/bash scripts, and read-only DB connections) that models already understand natively from pre-training.
Pillar 2: From Ad-Hoc Chat to Plan-First & Background Agent Workflows
Addresses: Time-consuming back-and-forth chat & over-engineering on complex prompts (Nina), moving from "AI-second" to "AI-first" (Ben), low adoption of agents beyond early adopters (Raj).
- The
Explore → Plan → Constrain → ExecuteLoop: Formalize the workflow recommended by the Cursor team (Raj) and used by power users (Ben):- Explore & Reproduce: Use AI code search and chat to map coupling and hidden side effects before touching code.
- Plan in Small Increments: Force the model to output a concise implementation plan and file-touch list before generating code—preventing the "complex prompt → over-engineered mess" failure mode Nina identified.
- Execute Incrementally: Apply changes step-by-step with human review at boundaries.
- Practical Delegation to Background Agents & Subagents: Demystify background agents by teaching engineers which tasks to delegate vs. keep local:
- High-ROI Background Agent Tasks (Delegate Now): Expanding test coverage on existing suites, removing stale feature flags, writing one-off DevOps/AWS/data scripts, initial frontend component scaffolding, and bug reproduction (Ben & Nina).
- Keep Human-in-the-Loop (Do Not Autonomously Delegate): Multi-party financial/invoicing logic, cross-service schema migrations, and regulated-data surfaces.
Pillar 3: Verification Guardrails & Compliance in High-Stakes Code
Addresses: "Plausible-looking" AI code introducing subtle bugs & weak E2E tests (Nina), correctness-critical business logic (Ben), 20–30 min CI / 2-hr deploys, regulated data & litigation risk from prior 3rd-party data incident (Raj).
- Test-First AI Development (Closing the Verification Gap): When working in coupled monolith code (like Ben's transaction/invoicing lifecycle) or with slow 30-minute CI pipelines (Raj), teach engineers to use AI to build the safety net before changing production code:
- Generate characterization tests and integration tests that lock in current behavior first.
- Use AI to generate local reproduction scripts and fast local test runners so errors are caught in seconds rather than waiting 30 minutes for CI or 2 hours for deploy.
- Reviewing as an "Editor, Not Author": Since Northcove engineers rarely write >20 lines by hand anymore (Raj), train them on an AI Code Review Checklist specifically targeting AI failure modes: hallucinated edge-case handling, silent coupling breakage, N+1 queries/migration locks, and over-abstracted boilerplate.
- Compliance & Regulated Data Red-Lines: Address senior engineer skepticism head-on with clear, non-negotiable engineering guardrails:
- How organizational data-isolation contracts work (e.g., Anthropic enterprise/zero-retention setup vs. unapproved third-party tools).
- Strict rules on sanitizing PII/regulated data in prompts, test fixtures, and MCP database connections to prevent any repeat of Northcove's prior third-party data-sharing incident.
Pillar 4: AI-First Scoping, Visual Prototyping, & Async Knowledge
Addresses: Bloated tech specs & alignment delays (Ben), thin PRDs, meeting overload for senior ICs, and cross-functional Cursor Slack usage (Raj, Nina).
- "Directionally Correct" Prototypes Over Bloated Tech Specs: Teach Ben's and Nina's PM's rapid prototyping workflow to tech leads and senior ICs. Instead of spending weeks writing and arguing over massive tech specs that conflate strategy, resourcing, and architecture:
- Use AI (Cursor throwaway sketches or Vercel
v0mockups) to generate visual, interactive prototypes in hours that don't need to compile or merge, just align stakeholders on what needs to be built. - Separate lightweight strategic alignment/prototyping from focused technical design docs.
- Use AI (Cursor throwaway sketches or Vercel
- Self-Documenting Workflows to Cut Meeting Load: Train engineers to use AI at the end of a PR or debugging session to automatically update in-repo architecture docs and summarize findings—reducing the synchronous meeting burden on senior ICs (Raj) and fixing the stale documentation problem (Nina/Raj).
- Enabling Non-Engineering Safely (Cursor Slack & BI): Establish lightweight patterns (and repo rules) so when PMs, Ops, or Marketing use the popular Cursor Slack integration to ask codebase/copy questions (Ben), the codebase is structured to give accurate answers without requiring a senior engineer in a meeting.
Pillar 5: Applied AI Product Engineering — Evals & Automated Prompt Optimization
Addresses: Nina's manual labeling (100–250 docs), labor-intensive prompt tweaking, and non-deterministic regressions on the LLM document interpretation system; Ben's insight on DSPy.
- Systematic Eval Harnesses (Moving Beyond Manual Spot-Checking): Using Nina's document-interpretation system as a real case study, train product-AI engineers on building repeatable evaluation pipelines:
- Structuring golden datasets (turning Nina's 100–250 labeled documents into automated regression suites).
- Combining deterministic schema/field assertions with calibrated LLM-as-a-Judge and production proxy metrics.
- Programmatic Prompt Optimization (
DSPy): Bridge Ben's awareness of DSPy with Nina's day-to-day pain. Show engineers how to stop hand-tweaking brittle prompt strings (where small edits cause non-deterministic regressions across varying document layouts) and instead use programmatic prompt compilation/optimization against their labeled ground-truth dataset.
3. Traceability Matrix: Interview Signals → Training Modules
| Engineer | Key Pain Points & Observations | Existing Strengths to Leverage | Training Pillar / Action |
|---|---|---|---|
| Nina Park(Full-Stack, >1 yr; AI Feature Builder & "Basic" Cursor User) | • Complex prompts lead to over-engineering & legacy antipatterns.• Back-and-forth chat is slow; wants background agents.• "Plausible" AI code hides subtle bugs; weak E2E/integration tests.• Hand-tweaking prompts & labeling 100–250 docs for LLM doc parser is grueling and non-deterministic. | • Effective at using concrete codebase examples to guide Cursor.• Hands-on experience building v2 production Anthropic LLM features & ground-truth datasets.• Team uses Vercel v0 for rapid UX prototyping. |
Pillar 1: Golden examples & .cursor/rules to stop legacy pattern drift.Pillar 2: Incremental Plan → Execute & background agent delegation.Pillar 3: AI-generated integration/E2E tests before code changes.Pillar 5: Automated evals & DSPy prompt optimization. |
| Ben Walsh(Senior SWE, Business Group; AI Power User) | • Can't use AI for core production features due to monolith coupling & startup antipatterns.• Quarterly tech specs are bloated and painful.• MCP servers (Jira/Confluence/GitHub) had poor performance.• Org is still "AI-second"; lacks standardized rules/prompts. | • Power user in #cursor-nerds & #background-agents (test coverage, flag removal, scripts).• Uses throwaway AI prototypes for fast alignment.• Strong mental models: "Context is king", "Directionally correct", CLI > MCP, awareness of DSPy. |
Co-Instructor / Champion: Have Ben co-lead sessions on upfront context tagging, CLI-driven workflows, background agents for test coverage/flag cleanup, and disposable prototyping for quarterly planning. |
| Raj(Engineering Org / Leadership Perspective) | • Experienced devs (4–10 YoE) are skeptical of AI on non-trivial work & DB migrations.• Prior 3rd-party data incident creates high compliance/litigation caution.• CI (20–30m) & deploys (40m–2h) are slow.• Siloed, stale docs & too many meetings draining senior ICs. | • High baseline Cursor adoption ("editors not authors", <20 lines written manually).• Fast 1-week bet cycles & strong engineering autonomy.• Cursor "Plan in chat → Build" workflow already seeded. | Pillar 1: Prescriptive custom rules for Northcove patterns & DB migrations.Pillar 3: Compliance red-lines & local test harnesses to beat 30m CI.Pillar 4: Async repo-docs & prototype-driven scoping to cut meeting load. |
4. Recommended Rollout & Workshop Structure
To respect senior engineers' time (given meeting fatigue) and overcome skepticism among 4–10 YoE developers, avoid long theoretical lectures. Deliver the training as role-targeted, hands-on labs inside Northcove's actual codebase, co-led by internal champions like Ben and members of #cursor-nerds.
Phase 0: Pre-Training Foundation (1 Week Before Workshops)
Before running training, partner with Ben and 2–3 engineers from Consumer, Business, and Platform to create a "Northcove AI Starter Kit" in the monolith:
- Base
.cursor/rulesPack: Codify Python/TypeScript conventions, prohibited legacy antipatterns, and database migration safety rules. - Canonical "Golden Examples" Index: A lightweight
AI_CONTEXT.mdmapping common tasks (e.g., API endpoint, React component, invoice calculation, DB migration) to 2–3 exemplary files in the repo. - Compliance & Data Isolation One-Pager: Clear rules on what data can/cannot touch Cursor, Anthropic APIs, and local CLI/MCP tools.
Module 1: Core AI Engineering in the Northcove Monolith (All 100 Engineers — 90 mins, Hands-On)
- Part A — Context Engineering (30 mins): Why AI fails on coupled Northcove code and how to fix it. Live demo by Ben on "Context is King" (file tagging, Golden Examples,
.cursor/rules, and why CLI tools beat slow MCP servers). - Part B — The
Plan → Constrain → VerifyWorkflow (35 mins): Moving from naive prompting to multi-step planning in Cursor Chat, generating characterization/integration tests first (solving Nina's "plausible code" risk and Raj's 30-min CI bottleneck), and editing AI output critically. - Part C — Compliance & High-Risk Boundaries (25 mins): Navigating database migrations, regulated data, and complex business logic safely.
Module 2: Background Agents, Subagents & Automation (All Engineers — 60 mins, Hands-On)
- Focus: Moving from tedious synchronous back-and-forth chat (Nina's pain) to asynchronous delegation.
- Lab Exercises:
- Dispatching a background agent to backfill integration test coverage on an undertested module.
- Using a background agent / subagent to rip out a deprecated feature flag or build an internal DevOps/support script ("directionally correct value-add").
- Setting up reusable custom slash commands/prompts for team-specific workflows.
Module 3: AI-First Scoping, Prototyping & Async Context (Tech Leads, Senior ICs & PMs — 60 mins)
- Focus: Cutting synchronous meeting load (Raj) and replacing bloated multi-week tech specs (Ben).
- Workflows Covered:
- Rapid visual alignment using throwaway Cursor prototypes and Vercel
v0mockups during quarterly planning and 1-week product bets. - Turning Slack/PR tribal knowledge into living, in-repo documentation so Cursor (including the Cursor Slack integration used by PMs/Ops) has accurate, up-to-date context.
- Rapid visual alignment using throwaway Cursor prototypes and Vercel
Module 4: Production LLM Features — Evals & Prompt Optimization (Targeted Track for AI Feature Teams like Nina's — 90 mins)
- Focus: Engineering rigor for customer-facing AI features (e.g., Document Interpretation v2).
- Workflows Covered:
- Designing automated evaluation pipelines (golden datasets, deterministic vs. proxy metrics) to measure accuracy without manual bottlenecks.
- Hands-on introduction to DSPy / programmatic prompt optimization to eliminate brittle manual prompt tweaking and stabilize non-deterministic outputs across varied document formats.
5. Assumptions & Notes
- Assumed Audience Split: All ~100 engineers attend Modules 1 & 2 (grouped by Consumer, Business, and Platform silos so examples match their domain), while Module 3 targets Senior ICs/Tech Leads/PMs and Module 4 targets engineers actively building LLM-powered product features.
- Tooling Scope: Recommendations assume Northcove standardizes on its existing Cursor Enterprise + Claude/Anthropic stack rather-than introducing net-new vendor platforms, prioritizing repo-native rules and CLI workflows over external MCP servers based on Ben's performance findings and Raj's compliance constraints.