On September 14, 2026, researchers at Tokyo-based Sakana AI published what theoretical computer scientists are calling the most significant mathematical challenge to standard backpropagation in decades: Augmented Lagrangian Predictive Coding (ALPC).
Since Rumelhart, Hinton, and Williams popularized the backpropagation algorithm in 1986, virtually every modern deep learning model—from dense convolutional networks to trillion-parameter mixture-of-experts transformers—has relied on the chain rule to propagate error gradients from the terminal loss function backwards through every layer. While immensely successful, backpropagation suffers from fundamental physical and architectural flaws known as backward locking and the weight transport problem.
In standard backpropagation, no hidden layer can compute its parameter updates until the entire forward pass has completed and the backward gradient wave propagates down from the output. Furthermore, GPUs must retain every single intermediate activation tensor in costly high-bandwidth memory (HBM) throughout the forward pass, creating the severe activation memory wall that dominates training cluster costs.
Sakana AI circumvents this four-decade-old bottleneck by introducing Augmented Lagrangian Predictive Coding (formally codified in the open-source pc-alm repository). Rather than treating the neural network as a rigid directed acyclic graph where gradients flow sequentially, ALPC conceptualizes the network as a dynamical physical system with continuous local energy states. Each layer maintains its own internal activity vectors and computes updates based strictly on local prediction errors relative to its immediate neighboring layers.
To prevent local energy minimization from diverging or collapsing into trivial degenerate solutions, Sakana AI couples predictive coding with the Augmented Lagrangian multiplier method from mathematical optimization. The multipliers act as rigid elastic cords between layers, enforcing exact forward-mapping consistency while allowing each layer to optimize its synaptic weights completely in parallel.
Empirical evaluations presented in the paper demonstrate that ALPC matches AdamW-optimized backpropagation on vision transformers (ViT) and autoregressive language modeling benchmarks (WikiText-103) within 0.1% perplexity, while reducing peak training activation memory by 68%. Because layers no longer wait for global backward passes, training can be distributed asynchronously across heterogeneous compute nodes without pipeline bubbles.
The discovery provides the long-sought mathematical bridge between biological neuroscience—where human brains learn continuously through local synaptic plasticity without global error backpropagation—and large-scale artificial intelligence, laying the groundwork for analog neuromorphic processors that train at a fraction of today's megawatt datacenter budgets.