Micrograd

Micrograd is a minimal automatic-differentiation engine written in Python by andrej-karpathy. It implements backpropagation in approximately 100 lines of code (engine.py) and includes a tiny neural-network library on top (nn.py, ~50 lines). It is designed for learning, not production.

---

Files

File	Contents
engine.py	Value class — scalar wrapper with _backward, gradient accumulation, topological sort
nn.py	Neuron, Layer, MLP — the neural network API on top of Value

---

The Value Class

The Value object is the atom of computation:

• data — the scalar numeric value
• grad — the accumulated gradient (initialised to 0)
• _backward — a closure that applies the chain-rule for this operation
• _prev — child nodes in the computational-graph

Operations on Value objects return new Value objects with _backward set correctly, building the graph implicitly.

---

Significance

Karpathy’s claim: micrograd is “all you need to train your networks.” It demonstrates that the core of deep learning — backpropagation, gradient-descent, multi-layer-perceptron architecture — is conceptually compact. Production libraries (pytorch, JAX) extend this to tensors and hardware acceleration without changing the mathematics.

---

Sources

• karpathy-2022-micrograd-backpropagation — primary; the lecture builds micrograd live from scratch