two karpathy talks, one mental model
notes on the 1hr intro and the 3.5hr deep dive, and what stuck after building on llms daily.
i spend most days building on top of llms now. agent session tooling, data pipelines, the usual. at some point using them daily without knowing how they work started to feel like driving without knowing what an engine does. so i watched both karpathy talks back to back: the 1hr intro from 2023 and the 3.5hr deep dive from 2025.
here is the whole thing compressed into one mental model, plus the three ideas that actually changed how i build.
tl;dr
- an llm predicts the next token over internet-scale text. everything else is scaffolding on that.
- pretraining builds the base model. post-training (chat data, human feedback, RL) turns it into an assistant.
- the models are jagged. brilliant in places, confidently wrong at counting and spelling.
- give them tokens to think and tools to lean on. never rely on mental arithmetic.
- RL on verifiable rewards is what gave us reasoning models. DeepSeek-R1 showed it works in the open.
the stack in three stages
| stage | what happens |
|---|---|
| pretrain | read the internet, learn to predict. FineWeb alone is 15 trillion tokens. GPT-2 trained on 100 billion, which counts as tiny now. |
| post-train | conversations first, then human preferences, then verifiable rewards. this is where the assistant comes from. a base model only completes text. |
| use | inference with tools, memory, and room to think. the psychology of the model at runtime matters as much as the weights. |
jagged intelligence
the failures do not look like human failures. a model that writes a flawless essay cannot count letters in a word. karpathy’s example: ask it to count, and the fix is to have it call a python routine. the interpreter does the counting, not the model’s mental arithmetic.
once you see this you stop being surprised and start designing around it. every sharp edge is a place where a tool call belongs.
RL actually working
the R1 section is the one i keep thinking about. math accuracy climbs as training runs, the thinking traces get longer, and new strategies show up unprompted. the aha moments are not programmed in. they fall out of optimizing a verifiable reward.
the AlphaGo parallel lands it. move 37 was a move no human expert would play, rated 1 in 10,000, brilliant in retrospect. imitation never finds that move. search plus a reward does.
the llm os, and its security problem
the intro talk frames chat assistants as an emerging operating system. the model is the kernel, tools are the apps, and the market splits the familiar way: proprietary systems on one side, a maturing open-source ecosystem led by llama on the other, like windows versus linux.
the same talk explains why the security problems are structural. prompt injection and jailbreaks are not bugs in the usual sense. when data and instructions share one channel, attacks come with the architecture. worth remembering every time you wire an agent to a real tool.
what i do differently now
two things in my own projects trace straight back to these talks. pi-resume-harness treats old tool output as stale and re-verifies the files before acting. that is the never-trust-mental-arithmetic rule, applied to agents. and repoatlas cites real files instead of trusting generated paths, because model memory is lossy and the file is not.
the third is smaller and daily. when a task needs exactness, i make the model write it down or call a tool. tokens to think, literally.
links
1hr intro 3.5hr deep dive tiktokenizer transformer visualizer llm.c gpt-2 repro
thanks for reading.