AI Anti-Patterns

Recognizing what NOT to do is as important as knowing best practices. This chapter catalogs common mistakes in AI system design.

Table of Contents

• Architecture Anti-Patterns
• RAG Anti-Patterns
• Agent Anti-Patterns
• Prompting Anti-Patterns
• Evaluation Anti-Patterns
• Production Anti-Patterns
• Interview Questions

---

Architecture Anti-Patterns

The God Prompt

Problem: Single massive prompt trying to do everything.

# ANTI-PATTERN: God Prompt
SYSTEM_PROMPT = """
You are a helpful assistant. You can:
1. Answer questions about our products
2. Help with technical support
3. Process refunds
4. Schedule appointments
5. Translate languages
6. Write code
7. Analyze data
8. Generate reports
... [continues for 5000 tokens]
"""

Why it fails:

• Context consumed by instructions, not user content
• Model struggles with conflicting instructions
• Impossible to optimize for all cases
• Updates affect everything

Solution:

# PATTERN: Specialized components
class QueryRouter:
    async def route(self, query: str) -> str:
        intent = await self.classify_intent(query)
        handler = self.handlers[intent]
        return await handler.process(query)

---

Single Provider Dependency

Problem: Entire system depends on one LLM provider.

# ANTI-PATTERN: Single provider
async def generate(prompt: str) -> str:
    return await openai.chat.completions.create(...)

Why it fails:

• Provider outage = complete system failure
• Rate limits affect all traffic
• No price negotiation leverage
• Locked into one model family

Solution:

# PATTERN: Multi-provider with failover
class LLMClient:
    def __init__(self):
        self.providers = [OpenAI(), Anthropic(), Google()]
    
    async def generate(self, prompt: str) -> str:
        for provider in self.providers:
            try:
                return await provider.generate(prompt)
            except ProviderError:
                continue
        raise AllProvidersFailedError()

---

Premature Fine-Tuning

Problem: Fine-tuning before exhausting simpler approaches.

Why it fails:

• Expensive and time-consuming
• Requires quality training data (often unavailable)
• Hard to update and maintain
• Often unnecessary

Decision flow:

Try prompting first
    ↓ (not working)
Try few-shot examples
    ↓ (not working)
Try RAG for knowledge
    ↓ (not working)
Consider fine-tuning (with 500+ examples)

---

RAG Anti-Patterns

Retrieve Everything

Problem: Retrieving too many documents regardless of relevance.

# ANTI-PATTERN: Retrieve everything
results = vector_db.search(query, top_k=50)
context = "\n".join([r.text for r in results])

Why it fails:

• Noise drowns out signal
• Exceeds context limits
• Wastes tokens on irrelevant content
• "Lost in the middle" effect

Solution:

# PATTERN: Quality over quantity
results = vector_db.search(query, top_k=20)
reranked = await reranker.rerank(query, results)
context = "\n".join([r.text for r in reranked[:5] if r.score > 0.7])

---

No Chunking Strategy

Problem: Arbitrary or no chunking of documents.

# ANTI-PATTERN: Fixed-size blind chunking
chunks = [text[i:i+1000] for i in range(0, len(text), 1000)]

Why it fails:

• Breaks mid-sentence, mid-paragraph
• Loses semantic coherence
• Separates related information
• Poor retrieval quality

Solution:

# PATTERN: Semantic-aware chunking
chunks = semantic_chunker.chunk(
    text,
    chunk_size=500,
    overlap=100,
    respect_boundaries=["paragraph", "section"]
)

---

Ignoring Metadata

Problem: Treating all documents as equal text.

# ANTI-PATTERN: Ignore metadata
embedding = embed(document.text)
vector_db.insert(embedding, {"text": document.text})

Why it fails:

• Cannot filter by date, source, type
• No access control per document
• Cannot weight recent vs old
• Loses valuable context

Solution:

# PATTERN: Rich metadata
vector_db.insert(embedding, {
    "text": document.text,
    "source": document.source,
    "date": document.date,
    "access_level": document.access_level,
    "document_type": document.type,
    "section": document.section
})

# Filter query
results = vector_db.search(
    query,
    filter={"date": {"$gte": "2024-01-01"}, "access_level": user.level}
)

---

Agent Anti-Patterns

Infinite Loop Risk

Problem: No termination conditions for agents.

# ANTI-PATTERN: No limits
while not done:
    action = await agent.decide_action()
    result = await execute(action)
    done = agent.check_done(result)

Why it fails:

• Agents can loop forever
• Costs spiral out of control
• Never returns to user
• Resource exhaustion

Solution:

# PATTERN: Multiple termination conditions
MAX_STEPS = 20
MAX_COST = 10.0
MAX_TIME = 300  # seconds

for step in range(MAX_STEPS):
    if cost_tracker.total > MAX_COST:
        return "Cost limit reached"
    if time.time() - start > MAX_TIME:
        return "Time limit reached"
    
    action = await agent.decide_action()
    result = await execute(action)
    
    if agent.check_done(result):
        return result
    
return "Step limit reached"

---

Unsafe Tool Access

Problem: Giving agents unrestricted tool access.

# ANTI-PATTERN: Full access
tools = [
    delete_file,
    execute_shell_command,
    send_email,
    database_query  # unrestricted!
]

Why it fails:

• Agent can delete critical files
• Can exfiltrate data
• Can execute malicious commands
• No audit trail

Solution:

# PATTERN: Scoped, validated tools
tools = [
    ScopedFileTool(allowed_dirs=["/tmp/agent"]),
    RestrictedShellTool(allowed_commands=["ls", "cat"]),
    EmailTool(requires_confirmation=True),
    ReadOnlyDatabaseTool(allowed_tables=["products"])
]

---

Agent Without Memory

Problem: Agent restarts from scratch every turn.

# ANTI-PATTERN: Stateless agent
async def handle_message(message: str) -> str:
    return await agent.run(message)  # No context

Why it fails:

• Cannot do multi-turn tasks
• Repeats same mistakes
• Cannot learn from experience
• Poor user experience

Solution:

# PATTERN: Persistent memory
async def handle_message(session_id: str, message: str) -> str:
    memory = await memory_store.get(session_id)
    response = await agent.run(message, memory=memory)
    await memory_store.update(session_id, memory)
    return response

---

Prompting Anti-Patterns

Vague Instructions

Problem: Ambiguous prompts expecting specific behavior.

# ANTI-PATTERN: Vague
prompt = "Help the user with their request."

Why it fails:

• "Help" is undefined
• No format specified
• No boundaries
• Inconsistent behavior

Solution:

# PATTERN: Specific and structured
prompt = """
You are a customer support agent for TechCorp.

Your role:
- Answer questions about our products
- Help troubleshoot issues
- Escalate to human when unsure

Response format:
1. Acknowledge the issue
2. Provide a solution or ask clarifying questions
3. Offer next steps

Do NOT:
- Make promises about refunds (escalate instead)
- Provide legal or medical advice
- Share internal company information
"""

---

No Output Format

Problem: Expecting structured output without specifying format.

# ANTI-PATTERN: Hope for structure
prompt = "Extract the person's name, date, and location from this text."
response = await llm.generate(prompt)
# Response: "The person is John, he was there on March 5th in NYC"
# Now try to parse that...

Solution:

# PATTERN: Explicit format
prompt = """
Extract information and return as JSON:
{
    "name": "string",
    "date": "YYYY-MM-DD",
    "location": "string"
}

Text: ...
"""
# Or use structured output APIs
response = await llm.generate(prompt, response_format={"type": "json_object"})

---

Evaluation Anti-Patterns

Vibes-Based Evaluation

Problem: "It looks good to me" as the evaluation method.

# ANTI-PATTERN: Manual spot-checking
for i in range(5):
    response = await generate(test_prompts[i])
    print(response)  # Developer looks at it
# "Looks good, ship it!"

Why it fails:

• Not reproducible
• Cherry-picked examples
• No baseline comparison
• Misses edge cases

Solution:

# PATTERN: Systematic evaluation
eval_dataset = load_eval_set()  # 100+ examples
results = []

for example in eval_dataset:
    response = await generate(example["input"])
    score = await evaluate(response, example["expected"])
    results.append(score)

metrics = {
    "accuracy": sum(results) / len(results),
    "failures": [e for e, r in zip(eval_dataset, results) if r < 0.5]
}

---

Training on Test Set

Problem: Using evaluation data for development decisions.

# ANTI-PATTERN: Overfitting to eval
for iteration in range(100):
    accuracy = evaluate_on_test_set()  # Same set every time
    tweak_prompt_based_on_failures(test_set)  # Optimizing for test set

Why it fails:

• Overfits to specific examples
• Real-world performance differs
• No true measure of generalization

Solution:

# PATTERN: Proper data splits
dev_set = load_dev_set()      # For iteration
test_set = load_test_set()    # Final evaluation only

# Iterate on dev set
for iteration in range(100):
    accuracy = evaluate(dev_set)
    improve_based_on(dev_set)

# Final evaluation on untouched test set
final_accuracy = evaluate(test_set)

---

Production Anti-Patterns

No Rate Limiting

Problem: Unlimited LLM calls per user.

# ANTI-PATTERN: Open access
@app.route("/generate")
async def generate():
    return await llm.generate(request.prompt)  # No limits!

Why it fails:

• Single user can exhaust budget
• Denial of service risk
• Cost surprises
• No fair usage

Solution:

# PATTERN: Rate limiting
@app.route("/generate")
@rate_limit(requests_per_minute=10, requests_per_day=100)
@cost_limit(max_cost_per_day=1.0)
async def generate():
    return await llm.generate(request.prompt)

---

No Caching

Problem: Every identical request hits the LLM.

# ANTI-PATTERN: No cache
async def answer_faq(question: str) -> str:
    return await llm.generate(question)  # Same FAQ, same cost every time

Why it fails:

• Wasted money on identical queries
• Unnecessary latency
• Inconsistent answers to same question

Solution:

# PATTERN: Semantic caching
async def answer_faq(question: str) -> str:
    cached = await cache.get_similar(question, threshold=0.95)
    if cached:
        return cached.response
    
    response = await llm.generate(question)
    await cache.set(question, response)
    return response

---

Interview Questions

Q: What is the biggest anti-pattern you see in LLM applications?

Strong answer:

"The most damaging is the 'God Prompt' anti-pattern: a single massive prompt trying to handle every scenario.

Why it is common: It seems simpler to start with one prompt and add instructions as needs arise.

Why it fails:

• Context consumed by instructions, not user content
• Conflicting instructions confuse the model
• Cannot optimize for different use cases
• Changes have unpredictable side effects

The fix: Route to specialized handlers. Each handler has a focused prompt optimized for one task. The router itself can be simple (keyword-based) or smart (LLM-based for complex cases).

This applies beyond prompts. The general principle is: decompose complexity into specialized components rather than cramming everything into one monolith."

Q: How do you avoid agent runaway costs?

Strong answer:

"Multiple limits at different levels:

Per-request limits:

• Maximum steps (e.g., 20)
• Maximum tokens (e.g., 50K)
• Maximum time (e.g., 5 minutes)

Per-session limits:

• Daily token budget
• Daily cost cap

Per-user limits:

• Rate limiting (requests per minute/hour/day)
• Cost attribution and caps

Monitoring:

• Real-time cost tracking
• Alerts for anomalies (single request > $1)
• Circuit breaker if costs spike

Architecture:

• Cascade from cheap to expensive models
• Cache common operations
• Batch similar requests

The key is assuming the agent will try to run forever. Build in hard stops at every level. I have seen agents run up $1000 bills in minutes without proper limits."

---

Previous: Design Patterns