简体中文
EnglishWebsite review
This review may include AI-generated content and is not individually verified. Domain, certificate and other facts come from the observed data on this page.
What is Neural networks and deep learning?
Neural networks and deep learning is Michael Nielsen's free, book-length online introduction to how neural networks work and how they are trained. It is written as a structured course rather than a reference manual, and it explains both the mathematics and the code behind the ideas.
What it covers
The book builds up from fundamentals:
- Handwritten digit recognition as the running example, starting with simple networks and moving to practical results.
- Core building blocks: perceptrons, sigmoid neurons, network architecture, and learning by gradient descent.
- Backpropagation, including the four equations behind it and a matrix-based explanation of why it works.
- Improving training: cross-entropy cost, overfitting and regularization, weight initialization, and how to choose hyper-parameters.
- Deep learning topics: the vanishing gradient problem, convolutional networks, and progress in image recognition.
- A visual proof that networks can compute any function, plus an appendix on whether there is a simple algorithm for intelligence.
Who it suits
The main audience is someone who wants to understand neural networks from the inside: students, programmers moving into machine learning, and readers who prefer worked explanations over API tutorials. It rewards readers who are comfortable with basic calculus and linear algebra and who are willing to read code. Someone who only wants to train a model quickly with a library will find it slower than a hands-on tutorial.
How to use it
Read the early chapters in order—digits, gradient descent, and backpropagation form the foundation for everything later. If you already know the basics, the chapters on regularization, hyper-parameters, and convolutional networks are the most useful standalone sections.
For a broader, more theoretical treatment, the site points readers to the Deep Learning book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville at Deep Learning. Readers who prefer video lectures may also value Andrew Ng's courses at Coursera.
Is Neural networks and deep learning free to read, and how can I donate?
Yes. Neural Networks and Deep Learning by Michael Nielsen is free to read online at Neural Networks and Deep Learning. The site presents the full book—from perceptrons and gradient descent through backpropagation, regularization, and convolutional networks—without a paywall or signup.
Donating
The book is donation-supported rather than sold. The site's donation note suggests a small amount (it mentions $5 as a suggestion, with the amount left to you) and also lists a Bitcoin address for contributions. Donations are voluntary; nothing in the text is gated behind them.
Practical reading note
The book is written for readers who want to understand the mechanics, not just call a library: it includes proofs, exercises, and runnable code for handwritten-digit classification. If you are deciding whether to start, the first chapter's digit-recognition example is the natural test—if the mix of math and code there suits you, the later chapters on backpropagation and convolutional networks will too.
As a next step, pair it with a hands-on framework once you reach the code chapters, and keep the book's code repository (linked from the site's Resources section) open alongside the text.
Do I need prior machine learning experience to follow this book?
No. Neural Networks and Deep Learning is written as an introduction, and its chapter sequence starts from the basics: perceptrons and sigmoid neurons, then a simple network for classifying handwritten digits, before moving to gradient descent, backpropagation, regularization and convolutional networks. You can begin without prior machine learning experience.
What you do need is enough comfort with math and programming to follow worked derivations and code. The book discusses matrix operations, cost functions and calculus-based reasoning around backpropagation, and it includes implementation material. If terms like derivatives, vectors and matrices are unfamiliar, expect to pause and review them rather than skim.
A practical way to test readiness is to read the opening chapter on recognizing handwritten digits. If you can follow the idea of adjusting weights to reduce error, continue; if the notation is the main obstacle, strengthen that foundation first and return. Readers who already know machine learning can use the later chapters on the vanishing gradient problem and convolutional networks as a compact refresher, while complete beginners should treat the early chapters as the main course.
How does the book explain backpropagation and the vanishing gradient problem?
The book treats backpropagation as the practical engine of learning and the vanishing gradient as the main reason deep networks were historically hard to train. The two are connected: backpropagation computes gradients by propagating error backward through layers, and the vanishing gradient describes what happens to those gradients as they travel back through many layers.
Backpropagation
The book builds the explanation in stages rather than presenting the algorithm as a black box:
- It starts with a fast matrix-based way to compute a network's output, so the reader sees the forward pass clearly first.
- It states the two assumptions the cost function must satisfy for backpropagation to work.
- It introduces the Hadamard product, then presents the four fundamental equations behind backpropagation, with optional proofs.
- It describes the algorithm itself, shows the code, and discusses in what sense backpropagation is a fast algorithm.
- A closing "big picture" section explains what the equations mean conceptually, not just mechanically.
The emphasis is on understanding why the four equations hold, so the reader can reason about training rather than only run code.
The vanishing gradient problem
The book frames this as part of the broader question "why are deep neural networks hard to train?" It covers:
- The vanishing gradient problem itself.
- What causes it.
- Unstable gradients in deep nets and in more complex networks.
- Other obstacles to deep learning beyond gradients.
The vanishing gradient is presented as one of several reasons deep networks resist training, which is why the book later moves to techniques such as the cross-entropy cost function, regularization, and better weight initialization.
Who benefits and how to use it
This material suits readers who already know basic calculus and want to implement a network from scratch rather than call a library. A concrete path: read the backpropagation chapter alongside the handwritten-digit example, then implement the four equations in the book's code and compare your gradients against a numerical check. After that, read the vanishing gradient chapter and deliberately build a deep sigmoid network to watch gradients shrink across layers.
For a complementary treatment, Deep Learning by Goodfellow, Bengio and Courville covers optimization and gradient issues at greater mathematical depth, while the full text at Neural networks and deep learning is the more hands-on starting point.
Decision criterion: choose this book if you want to derive and code backpropagation yourself; look elsewhere if you mainly need a survey of modern architectures.
Can I run the book's example code to build a handwritten digit classifier?
Yes. The book is built around a worked handwritten-digit classifier, and it includes a code chapter for implementing and training that network, so you can follow along and run the examples rather than only reading the math.
What you get to work with
The table of contents shows a clear path from theory to running code:
- Perceptrons and sigmoid neurons — the basic building blocks.
- A simple network to classify handwritten digits — the task setup.
- Learning with gradient descent — how training works.
- Implementing our network to classify digits — the code.
- How the backpropagation algorithm works — including "The code for backpropagation."
- Improving the way neural networks learn — cross-entropy, regularization, weight initialization, and "Handwriting recognition revisited: the code."
- Convolutional networks in practice and "The code for our convolutional networks."
There is also a linked code repository, which is the natural place to get the scripts and data.
Practical next step
Start with the simple network chapter and its implementation, get the training loop running on the digit data, then move to the "Handwriting recognition revisited" code once you reach the improvement techniques. Treat the later convolutional-network code as a second stage, since it assumes you are comfortable with the earlier material.
Decision criteria
- If you want to understand the algorithm — read the backpropagation chapter alongside its code and step through the four fundamental equations.
- If you mainly want a working classifier — go straight to the implementation chapters and the repository, then return to the theory when results confuse you.
- If you are teaching or self-studying — the exercises and problems are part of the design, so budget time for them rather than only running the scripts.
For a broader treatment of the same subject, the book links to Deep Learning by Goodfellow, Bengio, and Courville, and the author's site is Neural networks and deep learning.
How do convolutional networks differ from the simple networks taught earlier in the book?
The book treats convolutional networks as a structural answer to a problem the earlier simple networks cannot solve well. The early chapters build fully connected networks: every neuron in one layer connects to every neuron in the next, and the network learns from scratch using backpropagation and gradient descent. That works for small tasks like classifying handwritten digits, but it scales poorly for images, because a fully connected first layer on a modest image already needs an enormous number of weights, and the network has no built-in notion that nearby pixels belong together.
What changes with convolutions
Convolutional layers replace full connectivity with small learned filters (kernels) that slide across the input. This introduces three practical differences:
- Local connectivity. Each unit sees only a small patch of the image rather than the whole input, matching the fact that edges and corners are local features.
- Weight sharing. The same filter is reused at every position, so the number of parameters drops sharply and the same feature detector works anywhere in the image.
- Pooling and hierarchy. Pooling layers shrink spatial dimensions, and stacked convolution-plus-pooling stages build from edges to parts to objects.
The earlier chapters also explain why deep plain networks train badly (the vanishing gradient problem), which is part of the motivation for architectures like convolutional networks rather than simply stacking more fully connected layers.
Choosing between them
| Situation | Simple fully connected network | Convolutional network |
|---|---|---|
| Input is a small feature vector (tabular data, tiny digit images) | Adequate and simpler to implement | Unnecessary overhead |
| Input is images or other grid-like data with spatial structure | Many parameters, ignores locality | Fewer parameters, exploits locality and translation |
| You are learning the fundamentals | The book's early chapters are the right starting point | Builds on the same backpropagation machinery |
| Compute and data are limited | Trains quickly on small data | Usually needs more data and compute to pay off |
A concrete reader scenario: someone working through the digit-recognition chapters can get a working classifier with a plain network, then move to the convolutional chapter to see how the same task improves when the architecture respects image structure. As a next step, compare parameter counts between the two designs for the same input size; the gap usually makes the trade-off obvious before any training run.
Namecheap
GitHub, Inc.
User reviews (0)