CTO's 20-Minute AI Primer

Tokenization

Why your LLM sees 'ChatGPT' as 3 tokens, not 1 word

Your LLM doesn't read words. It reads tokens — subword fragments that determine what your model can understand, how much it costs per query, and why it sometimes misspells simple words.

Attention

Why your inference costs scale quadratically

Attention is the mechanism that lets each token look at every other token to understand context. It's why 'bank' means different things in 'river bank' vs 'bank account' — and why doubling your context length quadruples your compute cost.

Context Windows

The budget your model runs on

The context window is the total number of tokens your model can process in one call — input + output combined. It's not just a limit, it's a budget: every token you spend on instructions is a token you can't spend on reasoning.

RAG

Retrieval-Augmented Generation: when and how

RAG retrieves relevant documents and feeds them into the model's context window so it can answer questions using your data — without fine-tuning. It's the most common production AI architecture, and the most commonly broken one.

Agent Architecture

ReAct loops, tool use, and memory

An AI agent is a system that can reason about a goal, choose tools to use, observe results, and adapt its approach — in a loop. The ReAct pattern (Reason → Act → Observe → Repeat) is the dominant architecture. Each loop iteration = another API call = more cost.