What Topics Does Stanford CS336 Cover?

Stanford CS336 covers the full pipeline of building a language model from scratch: data collection and cleaning for pre-training, transformer model construction, model training, and evaluation before deployment. According to the official course site, the course is designed so that students develop their own language model end to end, drawing inspiration from operating systems courses that build an entire OS from scratch. It is listed as Stanford / Spring 2026, with previous offerings in Spring 2025 and Spring 2024.

The core idea: build the whole stack yourself

The course description frames language models as the cornerstone of modern NLP and a general-purpose paradigm for downstream tasks. Rather than treating a model as a black box, CS336 walks through every aspect of language model creation. The comparison to operating systems courses is the clearest signal of scope: you are expected to implement the major components yourself, not just call an existing library.

Topics covered

Based on the course description, the pipeline breaks into four stages:

• Data collection and cleaning for pre-training — assembling and preparing the raw text a model learns from.
• Transformer model construction — building the architecture itself.
• Model training — running and optimizing training, including making neural language models run quickly and efficiently on GPUs across multiple machines.
• Evaluation before deployment — assessing the model prior to putting it to use.

The through-line is that these are not surveyed in the abstract; they are steps in producing a working language model.

What the workload implies

The prerequisites section states that students receive minimal scaffolding and will write at least an order of magnitude more code than in other AI classes. That has a direct bearing on how the topics are taught: expect the data, transformer, training, and evaluation stages to be implementation-heavy rather than lecture-only. Proficiency in Python and software engineering is described as paramount, and the systems portion assumes strong familiarity with PyTorch and basic concepts like the memory hierarchy.

Where to go next

The course site organizes the material under Course Staff, Logistics, Content, Coursework, and Schedule, with a YouTube playlist for recordings. If you want to follow the topic sequence in order, the Schedule is the place to look; if you are deciding whether the course fits your background, start with the prerequisites above.