LLM Internals
The architectural core of modern LLMs: transformers, MoE, attention math, RoPE, GQA, KV cache, and the inference-optimal scaling shift driving 2026 model design.
Tokenization Deep Dive
Tokenization is the process of converting text into discrete units (tokens) that models can process. It directly impacts model capabilities, costs, and performance.
Attention Mechanisms
Attention is the core innovation that enables transformers. This chapter covers the mathematical foundations, variants, and optimizations that are essential for system design and interviews.
Transformer Architecture
This chapter provides a comprehensive view of the complete transformer architecture, bringing together the components from previous chapters into a unified understanding.
Embeddings and Vector Spaces
Embeddings are dense vector representations of text that capture semantic meaning. They are foundational to RAG systems, semantic search, and many AI applications.
Inference Pipeline
This chapter covers how LLMs generate text at inference time, the computational phases involved, and the key metrics for production serving.