Skip to main content

πŸ”„ Transitioning to AI Engineering Roles

A concrete, role-specific guide for engineers, PMs, QAs, and managers moving into AI-focused positions. No generic advice. Every path maps to real skills, real repo sections, and real courses.


Who This Guide Is For​

You currently work as a software engineer, QA, PM, EM, or data engineer, and you want to move into an AI-focused role. This guide maps your existing skills to specific AI roles, tells you exactly what gaps to close, and points you to the right sections of this repo and courses to fill them.


The AI Role Landscape​

Before picking a path, understand what the target roles actually are:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ AI ROLE LANDSCAPE β”‚
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ APPLICATION LAYER β”‚ β”‚ INFRASTRUCTURE LAYER β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ LLM App Engineer β”‚ β”‚ MLOps / AI Infra Engineer β”‚ β”‚
β”‚ β”‚ AI Product Engineer β”‚ β”‚ AI Platform Engineer β”‚ β”‚
β”‚ β”‚ Agentic Systems Eng β”‚ β”‚ AI Reliability Engineer β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ QUALITY LAYER β”‚ β”‚ LEADERSHIP LAYER β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ AI Eval Engineer β”‚ β”‚ AI Product Manager β”‚ β”‚
β”‚ β”‚ AI Quality Engineer β”‚ β”‚ AI Engineering Manager β”‚ β”‚
β”‚ β”‚ Red Team Analyst β”‚ β”‚ AI Program Manager β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ RESEARCH LAYER β”‚ β”‚ SPECIALIST LAYER β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ Applied AI Scientistβ”‚ β”‚ Agentic Coding Specialist β”‚ β”‚
β”‚ β”‚ Fine-tuning Engineerβ”‚ β”‚ RAG Architect β”‚ β”‚
β”‚ β”‚ Alignment Researcherβ”‚ β”‚ AI Safety Engineer β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Transition Paths by Current Role​


1. πŸ–₯️ Backend Engineer β†’ AI Engineering​

Why backend is the best starting point: You already understand APIs, latency, databases, distributed systems, and production reliability. AI applications need all of these. The gap is mostly domain knowledge, not engineering fundamentals.

Target Roles​

Backend Engineer
β”‚
β”œβ”€β”€β–Ί LLM Application Engineer (most common transition, 3–6 months)
β”œβ”€β”€β–Ί Agentic Systems Engineer (3–9 months)
β”œβ”€β”€β–Ί AI Infrastructure / MLOps Eng (6–12 months, needs GPU/serving knowledge)
└──► RAG Architect (4–8 months)

Skill Gap Analysis​

You already haveGap to closePriority
REST API designLLM API integration patternsπŸ”΄ High
Database designVector databases (Qdrant, Pinecone, Weaviate)πŸ”΄ High
Async/streamingStreaming LLM responses, token streamingπŸ”΄ High
Auth & multi-tenancyMulti-tenant RAG isolationπŸ”΄ High
Caching (Redis, CDN)Prompt caching, semantic caching🟑 Medium
Monitoring (Prometheus)LLM observability (traces, evals)🟑 Medium
CI/CDLLMOps pipelines, model version management🟑 Medium
N/AEmbedding models and vector math🟑 Medium
N/APrompt engineering fundamentals🟑 Medium
N/ARAG pipeline architecture🟑 Medium
N/AAgent frameworks (LangGraph, CrewAI)🟒 Lower
N/AFine-tuning concepts (LoRA, RLHF)🟒 Lower

Your 90-Day Plan​

Month 1: LLM Integration

  • Learn OpenAI / Anthropic API (streaming, function calling, structured output)
  • Build a simple RAG system: PDF ingestion β†’ Qdrant β†’ LLM response
  • Read this repo: 01-foundations, 02-model-landscape, 05-prompting-and-context
  • Course: ChatGPT Prompt Engineering for Developers (DeepLearning.AI, free)

Month 2: Production Patterns

Month 3: Agentic Systems

Portfolio Project Ideas​

  • Multi-tenant document Q&A service with access control
  • Agentic code reviewer that posts GitHub PR comments
  • RAG-powered internal knowledge base with eval pipeline

2. 🎨 Frontend Engineer β†’ AI Product Engineering​

Why this transition works: Frontend engineers understand UX, real-time UI updates, and user behavior. AI products live or die on UX β€” streaming responses, progressive rendering, loading states, feedback collection. Your skills are more valuable than you think.

Target Roles​

Frontend Engineer
β”‚
β”œβ”€β”€β–Ί AI Product Engineer (3–6 months β€” highest demand)
β”œβ”€β”€β–Ί AI UX Engineer (3–6 months, UX focus)
└──► Full-Stack LLM Engineer (6–9 months, add backend LLM skills)

Skill Gap Analysis​

You already haveGap to closePriority
Streaming UI (SSE, WebSocket)LLM token streaming integrationπŸ”΄ High
State managementConversation state, session memoryπŸ”΄ High
User feedback patternsAI feedback collection (thumbs, ratings)πŸ”΄ High
Form validationPrompt input validation and sanitization🟑 Medium
Error handling for asyncLLM timeout, fallback, retry patterns🟑 Medium
A/B testingLLM A/B testing and variant tracking🟑 Medium
N/ABasic prompt engineering🟑 Medium
N/ALLM API integration (at least one provider)🟑 Medium
N/AUnderstanding of context windows🟑 Medium
N/ABasic RAG concepts (what it is and why)🟒 Lower

Your 90-Day Plan​

Month 1: Integrate LLMs into UI

  • Build a streaming chat interface (Next.js + Vercel AI SDK)
  • Implement proper loading states, token-by-token rendering, error boundaries
  • Add a feedback widget (thumbs up/down, regenerate button)
  • Course: ChatGPT Prompt Engineering for Developers (DeepLearning.AI, free)

Month 2: UX Patterns for AI

Month 3: Eval Integration

  • Instrument your UI to collect feedback signals
  • Connect feedback to a Langfuse or LangSmith project
  • Run a basic A/B test between two prompt variants
  • Read this repo: 14-evaluation-and-observability
  • Course: Evaluating and Debugging Generative AI (DeepLearning.AI + W&B, free)

Portfolio Project Ideas​

  • Streaming document editor with AI suggestions and inline citations
  • Multi-step AI form wizard with persistent context
  • AI feedback dashboard showing per-feature quality metrics

3. πŸ§ͺ QA Engineer β†’ AI Eval Engineer​

Why QA is the most underrated path: AI evaluation is essentially a new form of QA. Manual test case design, edge case thinking, regression prevention β€” these are exactly what AI systems need. But the tools are different, and the mindset around non-deterministic outputs needs to shift.

Target Roles​

QA Engineer
β”‚
β”œβ”€β”€β–Ί AI Eval Engineer (3–6 months β€” best fit, fast transition)
β”œβ”€β”€β–Ί AI Quality Engineer (3–6 months)
└──► Red Team Analyst (6–9 months, security focus)

Skill Gap Analysis​

You already haveGap to closePriority
Test case designEval dataset creation (dimensional sampling)πŸ”΄ High
Regression testing mindsetEval suites as CI quality gatesπŸ”΄ High
Bug reportingError analysis methodology (open/axial coding)πŸ”΄ High
Test automationLLM-as-judge evaluator automationπŸ”΄ High
Non-functional testingHallucination, bias, toxicity detection🟑 Medium
User acceptance testingHuman annotation workflows🟑 Medium
N/ATracing and observability setup🟑 Medium
N/ARAGAS metrics (faithfulness, relevance, recall)🟑 Medium
N/ABasic prompt engineering🟒 Lower
N/APython scripting for eval pipelines🟒 Lower

Your 90-Day Plan​

Month 1: Error Analysis Foundation

  • Set up Langfuse or Phoenix tracing on any LLM application (your own or open source)
  • Do 3 rounds of manual error analysis: review 50 traces, write notes, categorize
  • Read the repo's evals companion guides:
  • Course chapter to read: Error Analysis: The Secret Sauce (inside evals guides, Chapter 3)

Month 2: Build Evaluators

  • Write 3 code-based evaluators (JSON schema check, format validator, regex-based)
  • Write 1 LLM-as-judge evaluator with Train/Dev/Test calibration
  • Introduce judgy for statistical bias correction
  • Read this repo: 14-evaluation-and-observability
  • Course: Quality and Safety for LLM Applications (DeepLearning.AI + WhyLabs, free)

Month 3: CI/CD Integration

  • Wire evaluators into a GitHub Actions workflow β€” eval runs on every PR
  • Define quality gates (faithfulness > 0.85, format pass rate > 0.99)
  • Create a weekly eval report dashboard
  • Course: Evals for AI (Maven, Hamel + Shreya β€” paid, worth it for career transition)

Portfolio Project Ideas​

  • Open-source eval suite for a public LLM application
  • Blog post: "How I applied QA methodology to catch LLM failures"
  • Eval pipeline template repo with LangSmith + GitHub Actions

4. πŸ“‹ Product Manager β†’ AI Product Manager​

Why PMs are uniquely positioned: AI products fail not because of bad models but because of bad product decisions (wrong problem, wrong eval criteria, wrong success metrics). PMs who understand AI failure modes are extremely rare and highly valued.

Target Roles​

Product Manager
β”‚
β”œβ”€β”€β–Ί AI Product Manager (3–6 months β€” direct analog)
β”œβ”€β”€β–Ί AI Program Manager (3–6 months, coordination focus)
└──► Head of AI Product (9–18 months, leadership path)

Skill Gap Analysis​

You already haveGap to closePriority
User researchError analysis as voice-of-customerπŸ”΄ High
Success metrics definitionAI-specific metrics (faithfulness, completion rate)πŸ”΄ High
Roadmap prioritizationFailure mode prioritization from eval dataπŸ”΄ High
A/B testingLLM A/B testing design (prompt variants, models)πŸ”΄ High
Stakeholder communicationExplaining AI limitations to partners🟑 Medium
PRD writingAI system capability docs and constraint documentation🟑 Medium
N/AHow LLMs work at a high level (no code required)🟑 Medium
N/ARAG pipeline concepts🟑 Medium
N/ATracing / observability tools (Langfuse UI)🟑 Medium
N/APrompt engineering basics🟒 Lower

Your 90-Day Plan​

Month 1: Build Technical Vocabulary

  • Read this repo's foundations, WITHOUT skipping to code:
  • Course: AI for Everyone (Coursera, Andrew Ng, free) β€” designed for non-technical roles

Month 2: Own Error Analysis

  • Ask your engineering team to set up Langfuse or LangSmith
  • Personally review 100+ traces from your product β€” take notes, find patterns
  • Run an error analysis session with your team; lead the failure mode categorization
  • Read this repo: 14-evaluation-and-observability
  • Read: Chapter 3 (Error Analysis) in AI Evals Comprehensive Study Guide

Month 3: Define Your Eval Strategy

  • Write an "AI Quality Spec" for your product: define what good looks like for each feature
  • Work with engineers to instrument evals for those criteria
  • Set success metrics for your next quarter that include AI quality gates (not just user growth)
  • Course: Evals for AI (Maven, Hamel + Shreya β€” explicitly designed for PMs)

Skills That Make You Stand Out as an AI PM​

  • You've personally reviewed traces (most PMs delegate this)
  • You can define failure modes quantitatively, not just qualitatively
  • You can communicate the cost of quality improvements (prompt changes vs. model upgrades vs. fine-tuning)
  • You understand the difference between RAG, fine-tuning, and prompt engineering β€” and when each is appropriate

5. πŸ‘¨β€πŸ’Ό Engineering Manager β†’ AI Engineering Manager​

The EM transition is about leadership evolution: Technical literacy in AI is necessary but not sufficient. The key shift is managing non-deterministic systems, teams evaluating quality without ground truth, and a field that changes every 3–6 months.

Target Roles​

Engineering Manager
β”‚
β”œβ”€β”€β–Ί AI Engineering Manager (6–12 months)
β”œβ”€β”€β–Ί Director of AI Engineering (12–24 months)
└──► VP of AI / Head of AI (18–36 months)

What Changes as an AI EM​

Traditional EMAI EM additions
Sprint planningEval-driven iteration cycles
PR review standardsEval suite as the new "tests pass" bar
Hiring for backend/frontendHiring for LLM, vector search, evals expertise
Incident response for outagesIncident response for quality regressions
Roadmap with feature flagsRoadmap with model upgrade risks
Performance reviews based on deliveryPerformance reviews including AI quality ownership

Your 90-Day Plan​

Month 1: Technical Depth

Month 2: Process and Team Design

  • Redesign your team's definition of "done" to include eval gates
  • Build an eval culture: weekly trace reviews, quality metrics in retros
  • Define your AI incident runbook: what happens when hallucination rate spikes?
  • Read this repo: 13-reliability-and-safety, 14-evaluation-and-observability

Month 3: Strategy and Hiring

  • Define the AI skills matrix for your team: who has what, what's missing
  • Build an interview rubric for AI engineers (use 00-interview-prep as your source)
  • Set team-level AI quality OKRs for next quarter
  • Course: CS294 LLM Agents (Berkeley, free) β€” gives you the depth for strategy conversations

6. πŸ› οΈ DevOps / Platform Engineer β†’ MLOps / AI Infrastructure Engineer​

Why platform engineers thrive here: Kubernetes, CI/CD, observability, cost management, SLAs β€” you've done all of this. The AI-specific additions are GPU scheduling, model serving, and LLMOps pipelines.

Target Roles​

DevOps / Platform Engineer
β”‚
β”œβ”€β”€β–Ί MLOps Engineer (3–6 months)
β”œβ”€β”€β–Ί AI Infrastructure Engineer (6–9 months)
└──► AI Platform Engineer (9–12 months)

Skill Gap Analysis​

You already haveGap to closePriority
Container orchestration (K8s)GPU node pools, NVIDIA device pluginsπŸ”΄ High
CI/CD pipelinesLLMOps pipelines (model eval, deployment gates)πŸ”΄ High
Observability stacksLLM-specific metrics (token throughput, TTFT)πŸ”΄ High
Cost managementGPU cost optimization, spot instances for trainingπŸ”΄ High
Secret managementAPI key rotation for multiple LLM providers🟑 Medium
N/AvLLM / TGI for self-hosted model serving🟑 Medium
N/AModel versioning and registry🟑 Medium
N/AQuantization basics (GPTQ, AWQ, GGUF)🟑 Medium
N/ABasic prompt engineering to understand what you're serving🟒 Lower

Your 90-Day Plan​

Month 1: LLM Serving

  • Deploy vLLM locally serving Llama 3.3 7B or Qwen2.5-Coder
  • Add Prometheus metrics: tokens/sec, latency P50/P95/P99, queue depth
  • Set up auto-scaling based on request queue
  • Read this repo: 04-inference-optimization, 11-infrastructure-and-mlops
  • Course: Efficiently Serving LLMs (DeepLearning.AI + Predibase, free)

Month 2: LLMOps Pipeline

  • Set up LangSmith or Langfuse for trace collection
  • Build a CI/CD quality gate: eval suite runs before model deploy
  • Implement prompt version control (Langfuse prompt registry or DSPy)
  • Read this repo: 14-evaluation-and-observability

Month 3: Scale and Cost

  • Compare self-hosted vs. API cost at target volume (use pricing guide in repo)
  • Set up cost dashboards per model, per team, per feature
  • Implement graceful multi-provider failover
  • Course: ML Engineering for Production (MLOps) (Coursera, DeepLearning.AI)

7. πŸ“Š Data Engineer β†’ AI Data / Feature Engineer​

Why data engineers are essential: Training data is the competitive moat of AI systems. Data pipelines, quality, and freshness determine model performance more than architecture. Your skills are immediately applicable.

Target Roles​

Data Engineer
β”‚
β”œβ”€β”€β–Ί AI Data Engineer (2–4 months β€” fastest transition)
β”œβ”€β”€β–Ί Embedding Pipeline Engineer (3–6 months)
└──► Fine-tuning Data Specialist (4–8 months)

Skill Gap Analysis​

You already haveGap to closePriority
ETL pipelinesDocument ingestion pipelines for RAGπŸ”΄ High
Data quality checksEval dataset quality validationπŸ”΄ High
Schema designMetadata schema for vector databasesπŸ”΄ High
Streaming pipelinesReal-time embedding and index update🟑 Medium
N/AEmbedding model selection and batching🟑 Medium
N/AVector database operations (upsert, filter, ANN search)🟑 Medium
N/AChunking strategies for document types🟑 Medium
N/AAnnotation pipeline design for fine-tuning🟒 Lower
N/ARLHF preference data format🟒 Lower

Your 90-Day Plan​

Month 1: RAG Data Pipeline

Month 2: Eval Dataset Engineering

  • Build a test dataset using dimensional sampling (see evals guides)
  • Set up human annotation pipeline using Label Studio or Argilla
  • Track inter-annotator agreement; reject low-quality labels
  • Read: AI Evals Comprehensive Study Guide, Chapter 12 (Human Annotation)
  • Course: Finetuning Large Language Models (DeepLearning.AI, free)

Month 3: Advanced Data Engineering

  • Build a pipeline that turns production traces into fine-tuning examples
  • Implement embedding drift detection: alert when document distribution shifts
  • Benchmark 3 embedding models on your domain data
  • Read this repo: 03-training-and-adaptation

πŸ“Š Role Comparison Overview​

Role Months to Avg Salary Best Suited For
First Role (US, 2026)
────────────────────────────────────────────────────────
Backend 3–6 mo $170–220K LLM App / Agentic Engineering
Frontend 3–6 mo $150–190K AI Product / UX Engineering
QA 3–6 mo $140–180K AI Eval / Quality Engineering
PM 3–6 mo $160–200K AI Product Management
DevOps 3–6 mo $170–220K MLOps / AI Platform
Data Eng 2–4 mo $165–210K RAG Data, Fine-tuning Data
EM 6–12 mo $200–280K AI Engineering Manager

Salaries are US market estimates based on Levels.fyi and LinkedIn data, May 2026. Ranges vary significantly by company, location, and experience level.


πŸ—ΊοΈ Which Repo Sections Map to What​

Use this when you're ready to go deep:

TopicRepo SectionWhy
How LLMs work01-foundationsFoundation for everything else
Which model to use02-model-landscapeModel selection is a daily decision
Fine-tuning03-training-and-adaptationFor fine-tuning data specialist path
GPU serving / vLLM04-inference-optimizationMLOps / Platform path
Prompt engineering05-prompting-and-contextEveryone needs this
RAG pipeline06-retrieval-systemsBackend / Data Eng path
Agentic systems07-agentic-systemsBackend / Senior AI Eng path
Memory & state08-memory-and-stateAll engineers building agents
LangGraph, CrewAI, Claude Code09-frameworks-and-toolsPractical tool selection
Document parsing10-document-processingData Eng / RAG path
GPU infra, LLMOps11-infrastructure-and-mlopsDevOps / Platform path
Multi-tenant security12-security-and-accessBackend / PM path
Guardrails, red teaming13-reliability-and-safetyQA / Red Team path
RAGAS, LangSmith, evals14-evaluation-and-observabilityQA / PM / all roles
Design patterns15-ai-design-patternsSenior level preparation
Case studies16-case-studiesInterview prep, reference designs
Evals deep diveAI Evals Comprehensive GuideQA / PM path
AI Evals (LangWatch)AI Evals LangWatch GuideQA / Eval Eng path
Interview prep00-interview-prepAll roles
CoursesCOURSES.mdAll roles

Full details in COURSES.md

Your RoleFirst CourseSecond CourseThird Course
BackendChatGPT Prompt Engineering for Devs (DL.AI, free)Building & Evaluating RAG (DL.AI, free)AI Agents in LangGraph (DL.AI, free)
FrontendChatGPT Prompt Engineering for Devs (DL.AI, free)Building Systems with ChatGPT API (DL.AI, free)Evaluating & Debugging GenAI (DL.AI + W&B, free)
QAAI Evals Guide in this repo (free)Quality & Safety for LLM Apps (DL.AI, free)Evals for AI – Maven (Hamel + Shreya, paid)
PMAI for Everyone (Coursera, free)AI Evals Guide Chapter 3 (free)Evals for AI – Maven (Hamel + Shreya, paid)
DevOpsEfficiently Serving LLMs (DL.AI, free)Evaluating & Debugging GenAI (DL.AI + W&B, free)ML Engineering for Production (Coursera)
Data EngBuilding & Evaluating RAG (DL.AI, free)Finetuning LLMs (DL.AI, free)AI Evals Guide in this repo (free)
EMGenerative AI with LLMs (Coursera)AI Agents in LangGraph (DL.AI, free)CS294 LLM Agents (Berkeley, free)

DL.AI = DeepLearning.AI


Common Mistakes to Avoid​

  1. Skipping fundamentals β€” Jumping to LangChain before understanding what an embedding is leads to cargo-cult code you can't debug.

  2. Building before evaluating β€” Ship nothing without a way to measure quality. Define your eval criteria before writing the first prompt.

  3. Copying prompts without understanding them β€” Prompts are engineering decisions. Understand why each element is there.

  4. Ignoring costs until it's too late β€” Every API call has a price. Build cost tracking from day one. See 02-model-landscape/03-pricing-and-costs.md.

  5. Assuming the model is the bottleneck β€” In most production AI systems, the bottleneck is retrieval quality, prompt design, or data quality. The model is rarely the problem.

  6. Using "latest" in model version strings in production β€” Pin exact versions. Silent model updates will break your product.

  7. Over-agenting β€” Starting with a 5-agent system when a single well-prompted call would work. Start simple, add complexity only when needed.


How to Get Hired​

Build in public. The AI engineering job market rewards demonstrated work:

  1. GitHub portfolio: One polished end-to-end project beats 10 toy projects
  2. Write a blog post: Describing one real problem you solved and how (error analysis, eval pipeline, RAG latency fix)
  3. Contribute to open source: OpenHands, LlamaIndex, DSPy, RAGAS. Even documentation PRs get you noticed.
  4. Use this repo's interview prep: 00-interview-prep/01-question-bank.md has 128 questions with strong answers

What to say in interviews:

  • Name specific decisions: "I chose Qdrant over Pinecone because of X" (not "I built a RAG system")
  • Cite failure modes you've encountered and how you fixed them
  • Know at least one benchmark by heart (SWE-bench, RAGAS scores, TTFT for your serving setup)
  • Show you think about eval and cost, not just features

Part of the AI System Design Guide β€” maintained by ombharatiya