Andrej Karpathy
Andrej Karpathy is a Slovak-Canadian AI researcher and educator known for making deep learning accessible. He is a co-founder of OpenAI, was Director of AI at Tesla (overseeing Autopilot), and later returned to independent research and education. He is the creator of micrograd and the YouTube series Neural Networks: Zero to Hero.
Key Contributions
| Work | Description |
|---|---|
| micrograd | Scalar-valued autograd engine in ~100 lines of Python; built live in his Zero to Hero lecture series |
| Neural Networks: Zero to Hero | YouTube lecture series building neural networks from scratch — from derivatives to GPT |
| nanoGPT | Minimal, readable GPT implementation in PyTorch |
| CS231n | Stanford’s Convolutional Neural Networks for Visual Recognition course (taught ~2015–2017) |
Teaching Philosophy
Karpathy’s stated goal is pedagogy over production. micrograd deliberately operates on scalars rather than tensors to expose the mathematical core of backpropagation without engineering complexity. He argues that the core ideas — automatic-differentiation, chain-rule, gradient-descent — are “all you need to understand neural network training,” and that everything else (tensors, parallelism, frameworks) is efficiency.
Sources
- karpathy-2022-micrograd-backpropagation — lecture transcript of the micrograd session