Computational Graph
A computational graph is a directed acyclic graph (DAG) in which:
- Nodes represent values (scalars or tensors)
- Edges represent the operations that produced each value from its inputs
automatic-differentiation engines like micrograd and pytorch build this graph implicitly during the forward pass. backpropagation then traverses it in reverse (via topological sort) to compute gradients.
Key Properties
- Directed — data flows from inputs to outputs; gradients flow the other way
- Acyclic — no cycles; ensures well-defined topological ordering
- Dynamic (eager) — in PyTorch and micrograd, the graph is built at runtime, making it easy to use Python control flow
Sources
- karpathy-2022-micrograd-backpropagation — introduces the graph via the Value object and _children pointers