AI Architecture for Engineers

Tokenization

Why your LLM sees 'ChatGPT' as 3 tokens, not 1 word

Your LLM doesn't read words. It reads tokens — subword fragments that determine what your model can understand, how much it costs per query, and why it sometimes misspells simple words.

Embeddings

How meaning becomes math

Embeddings convert words, sentences, and documents into arrays of numbers where mathematical distance equals semantic similarity. 'King' minus 'Man' plus 'Woman' ≈ 'Queen' isn't a trick — it's how the math actually works.

Attention

Why your inference costs scale quadratically

Attention is the mechanism that lets each token look at every other token to understand context. It's why 'bank' means different things in 'river bank' vs 'bank account' — and why doubling your context length quadruples your compute cost.

Context Engineering

The new prompt engineering

Context engineering is the discipline of deciding what information goes into the context window, in what order, and at what priority. It's prompt engineering evolved — less about wording tricks, more about information architecture.

RAG

Retrieval-Augmented Generation: when and how

RAG retrieves relevant documents and feeds them into the model's context window so it can answer questions using your data — without fine-tuning. It's the most common production AI architecture, and the most commonly broken one.

Guardrails

How to cage an AI system

Guardrails are input/output filters that prevent your AI system from saying things it shouldn't: harmful content, hallucinated facts, leaked PII, off-topic responses, or actions outside its authority. They're the difference between a demo and a production system.

Agent Architecture

ReAct loops, tool use, and memory

An AI agent is a system that can reason about a goal, choose tools to use, observe results, and adapt its approach — in a loop. The ReAct pattern (Reason → Act → Observe → Repeat) is the dominant architecture. Each loop iteration = another API call = more cost.