What Is Neural Networks and Deep Learning?

Neural Networks and Deep Learning is a free online book by Michael Nielsen that teaches the foundations of neural networks and deep learning, from perceptrons and gradient descent through backpropagation and convolutional networks. It suits self-directed learners who want a conceptual, exercise-driven introduction and are comfortable reading code and math; it is a book, not a course, so there are no deadlines, credentials, or instructor support.

What the book covers

The site presents the material as a sequence of chapters, each building on the last:

• Using neural nets to recognize handwritten digits — perceptrons, sigmoid neurons, network architecture, learning with gradient descent, and implementing a classifier.
• How the backpropagation algorithm works — a matrix-based warm-up, the assumptions needed about the cost function, the four fundamental equations behind backpropagation, the algorithm itself, and its code.
• Improving the way neural networks learn — the cross-entropy cost function, overfitting and regularization, weight initialization, revisiting handwriting recognition in code, and choosing hyper-parameters.
• A visual proof that neural nets can compute any function — universality, with caveats and extensions beyond sigmoid neurons.
• Why deep neural networks are hard to train — the vanishing gradient problem, unstable gradients, and other obstacles.
• Deep learning — introducing convolutional networks, convolutional networks in practice, code for a convolutional network, and recent progress in image recognition.
• On the future of neural networks, plus an appendix asking whether there is a simple algorithm for intelligence.

The progression is deliberate: you meet the ideas in a small, concrete setting (digit classification) before they are generalized.

How the material is structured for learning

Each chapter mixes explanation with two kinds of practice: exercises, which are embedded in the text and meant to be done as you read, and problems, which are more open-ended and appear at the end of sections. Code examples accompany the implementation chapters, and the site links to a code repository.

A reasonable working pattern, for example when studying backpropagation:

1. Read the conceptual sections and attempt the embedded exercises before looking ahead.
2. Work through the four fundamental equations and their (optional) proofs, checking each against the matrix formulation.
3. Read the backpropagation code alongside the algorithm description, matching each step to a line.
4. Attempt the end-of-section problems to test whether the ideas transfer.

The expected result is that you can explain why backpropagation is fast and implement a working network, rather than only recognize the terminology.

Practical details

The donation request is explicitly framed as optional and tied to the benefit you get from the book — it is not a paywall, and the text is presented for reading on the site.

When this book is the right choice

Choose it if you want a from-first-principles treatment where you write and reason about small networks yourself, and you are willing to do the exercises. It is less suitable if you want a current survey of state-of-the-art architectures, a certificate, or a guided cohort — the book's own framing is foundational, and its "recent progress" discussion is a chapter rather than a living reference. Readers who prefer a broader academic treatment can use the linked Goodfellow, Bengio, and Courville book as a companion.

Where to start

Open the site and read "What this book is about" and "On the exercises and problems" first — they set expectations for how to use the chapters. Then begin with the handwritten-digit chapter and keep the code repository open in a second tab. If you want to know when Nielsen announces new projects, the site points to his project announcement mailing list.