Thematic
Nemotron 3.5 Super VlOpenCodehigh effortrun 16 Oct 202642,934 tokens
Instructions
Instructions — what the model was asked
help me synthesize what to focus on in our AI engineering training for this company based on meeting notes with three engineers
help me synthesize what to focus on in our AI engineering training for this company based on meeting notes with three engineers
ANSWER.mdthe model's output · 6.98 KBdownload
AI Engineering Training Focus — Northcove
Synthesized from meeting notes with Nina (full stack), Ben (Senior SWE, Business group), and Raj (Engineering org overview). Company: Northcove. Stack: Python backend, TypeScript frontend, monolithic app, ~100 engineers across Consumer/Business/Platform silos.
Top Training Priorities (ranked)
1. Context Engineering & Company-Specific Knowledge (highest priority)
Every engineer cited context as the #1 blocker to effective AI use.
- The problem: Documentation is scattered and stale (Google Docs, Confluence, Slack threads). There's no structured way to feed documentation as context to Cursor. AI is "not good at Northcove-specific implementations." Ben's core principle: "Context is king — don't just say 'do the thing', tag relevant files upfront."
- Train on:
- How to build and maintain a company-specific context library (reusable prompts, rules,
.cursorrules/equivalent, tagging relevant files) - Retrieval-augmented generation (RAG) patterns for internal docs: indexing Confluence/Slack/Google Docs so AI can ground answers in current Northcove standards
- Writing effective system prompts that encode Northcove coding standards, frameworks, and architectural conventions
- Using similar examples from the codebase rather than general prompts (Nina: better results with concrete examples)
- How to build and maintain a company-specific context library (reusable prompts, rules,
2. Evaluation, Testing & Guardrails for AI-Generated Code
- The problem: AI code "looks plausible but may introduce subtle issues." Nina lacks robust end-to-end and integration testing and monitoring. Her document-interpretation eval is labor-intensive manual labeling (100–250 docs) with non-deterministic results from small prompt changes. Ben avoids AI for production features because of hidden side effects and coupling. Raj is skeptical of AI for DB migrations and regulated-data work.
- Train on:
- Building ground-truth datasets and eval harnesses for LLM features (proxy metrics for scale, labeled samples for accuracy)
- Guardrails for AI-generated code: review checklists especially for database/regulated-data changes
- Integration/e2e testing strategies to catch subtle AI-introduced regressions
- Monitoring and validation systems for AI-driven features in production
- When not to use AI: DB migrations, regulated data, significant non-trivial features
3. Prompt Engineering Standardization & Optimization
- The problem: Each engineer writes ad-hoc prompts. Ben wants organizational buy-in for rules, commands, and reusable prompts. Nina struggles to create comprehensive system prompts. Ben is aware of DSPy for prompt optimization but hasn't implemented it.
- Train on:
- Standardized, Northcove-specific prompt templates and reusable commands
- Prompt iteration methodology (small changes → non-deterministic outcomes; measure before/after)
- Introduction to automated prompt optimization (DSPy-style) for teams with labeled data
- "Directionally correct is the name of the game" — knowing when approximate AI output is acceptable (internal scripts) vs. when it isn't (production, regulated)
4. AI-First Workflow & Agent Adoption
- The problem: Most of the eng org is still "AI-second" rather than AI-first. Ben is a power user; Nina is a basic user. Agents/background processes are under-adopted. Everyone wants to delegate larger features to independent agents and reduce back-and-forth.
- Train on:
- AI-first integration: using AI for questions, reproduction, documentation, planning, debugging — not just coding
- Cursor power-user patterns: smart autocomplete, chat-for-planning-then-building, code search for structure
- Background agents and subagents: test coverage, feature flag removal, debugging, long-running processes
- Prompting for incremental changes (not just generating from scratch)
- Moving from "editor" to effective AI collaborator without over-reliance
5. AI-Assisted Prototyping & Planning
- The problem: Tech spec documents try to solve strategic alignment, deadlines, resourcing, sequencing, and technical alignment all at once — too much, hard to review. Planning isn't prescriptive.
- Train on:
- Using AI-generated prototypes (Vercel v0-style) for alignment — directionally correct, cheap to throw away
- Splitting planning artifacts: separate strategic alignment from technical implementation details
- Rapid prototyping for user research and PM/engineer alignment
6. Compliance, Data Isolation & Regulated Data
- The problem: A prior third-party data-sharing incident required reworking paid/organic surfaces. Litigation concerns make the team cautious. Nina uses Anthropic with organizational data isolation contracts. Raj flags regulated-data exposure as a key risk.
- Train on:
- Data isolation contracts and what can/cannot be sent to AI models
- Handling regulated/PII data in AI workflows
- Vendor/model selection with compliance in mind
7. Cross-Functional AI Adoption (enablement)
- The problem: Non-engineering functions (PMs, BI) are adopting AI via Cursor Slack integration but lack foundation. BI hookup needed. User research via AI is "not ready" (only talking to 3 people, not enough volume).
- Train on:
- Foundational AI literacy for non-engineers so they can benefit safely
- Guidelines for PMs using AI for copy, change requests, and prototyping
- Where AI fits (and doesn't) in user research and BI
Suggested Training Structure
| Module | Audience | Format | Focus |
|---|---|---|---|
| AI-First Foundations | All engineers | Workshop + hands-on | Cursor power use, context tagging, "context is king" |
| Context & RAG for Northcove | All engineers | Lab | Indexing internal docs, custom rules/system prompts |
| Eval & Guardrails | Backend + AI feature owners | Lab | Labeled datasets, proxy metrics, review checklists, monitoring |
| Agents & Background Automation | Senior/intermediate | Workshop | Subagents, background tasks, incremental changes |
| AI Prototyping & Planning | PMs + tech leads | Workshop | v0-style mockups, split planning artifacts |
| Compliance & Regulated Data | All (mandatory) | Short session | Data isolation, regulated data, incident lessons |
| AI for Non-Engineers | PMs, BI, ops | Intro session | Safe AI use, Cursor Slack, copy/change requests |
Success Metrics to Track
- % of engineers using AI-first (vs AI-second) workflow
- Reduction in time from PRD to deploy for bets (currently 1 week)
- AI-generated code caught by review/guardrails before production
- Eval coverage for LLM-powered features (vs manual labeling only)
- Reduction in "context not found" / stale-document incidents
Assumptions Made
- Northcove uses Cursor as its primary AI IDE (confirmed across all three interviews).
- Anthropic is the preferred model provider (per Nina's document system) with data isolation contracts in place.
- Training should be role-differentiated: basic users (Nina) need foundations, power users (Ben) need advanced agent/DSPy content, org leads (Raj) need compliance and knowledge-management focus.