State-compression ladder · Note M14 · Part III
Two arcs asked whether a richer substrate helps, and found it doesn't. The intervention that finally worked wasn't a new neuron at all — it was moving where the sequential depth lives.
M13.5 showed a faithfully compressed neuron bought no accessible computation. M14 showed relational structure between units didn't rescue it under a real event interface. Both are substrate hypotheses, and both failed. Part III changes the axis of the question: not what the units are, but where the computational depth physically resides — in time, or in space.
Take one serialized 64-pixel digit and factor it as S × T = 64: S parallel lanes, each a copy of the same tiny shared recurrent cell, each running T ticks. At 1×64 the computation is almost all time — one cell, 64 recurrent updates, a back-propagation horizon of 64. At 64×1 it is almost all space — 64 cells, one update each, no temporal credit path. A fixed random permutation is applied first, so any sweet spot is about the space–time split, not rediscovered image topology. Only S/T changes; cell law, width, initialization, optimizer, data and update count are held identical.
Careful with the name: what this measures is the space↔time transpose — spatializing what would have been recurrent depth, and thereby shortening the BPTT horizon. It is not yet backprop-through-space itself: there is no deep spatial causal chain carrying credit. That experiment comes last.
The narrow, defensible statement:
On pSeqDigits-64 with the tested shared tanh cell, reducing temporal depth by a modest spatial replication removed most of the optimization deficit.
8×8 reaches ≈98% of the fully-spatial 64×1 accuracy — using one-eighth of its simultaneous recurrent state, and cutting the back-propagation horizon from 64 to 8.
The extremes bound it: pure time (T=64, ρG≈10−8, accuracy 0.60±0.03) is credit-starved; the plateau from 8×8 onward sits at 0.95–0.98. Crucially, the plateau is non-monotonic in hardware: 8×8 (128 state) matches 16×4 (256) and 32×2 (512). Doubling the state buys nothing.
Accuracy tracks gradient survival ρ_G — the fraction of the loss gradient still reaching the earliest state — and ρG is governed by the BPTT horizon T, not by state count. Once spatialization shortens the horizon enough to lift ρG out of the dead zone, accuracy saturates.
What we will not claim is that this proves temporal credit alone governs accuracy. Changing S alters several things at once — simultaneous state, classifier input width, and the temporal horizon together. The non-monotonic plateau is strong evidence against a simple "more spatial capacity is better" story, and ρG is a compelling mechanistic correlate, but it is not a formal isolation of gradient survival from every other consequence of factorization. The honest form:
The results identify temporal credit depth as the dominant constraint in this tested regime.
The successful intervention was architectural: shorten the temporal credit path by spending a modest amount of space. Not a more exotic substrate, not more hidden state, not phase relations — simply changing where sequential depth lives.
Spatialization shortened the temporal credit path and exposed a favourable space–time operating point. The remaining question is whether the resulting spatial credit path can itself be implemented cheaply — because that is the real backprop-through-space test, and the real hardware prize.
The architecture that follows the knee is blocks: a short, exact local recurrence inside each block, composed spatially between blocks.
8 ticks of exact BPTT inside each block + spatial credit between blocks
with a credit ladder to climb:
And a genuinely parallel escape hatch for the right primitives: an associative recurrence — like a selective state-space cell c₁=aₓc+bₓ — admits a parallel scan in O(log T) depth. That reframes the substrate question in a new and possibly more consequential way: which dynamical primitives are expressive enough for the task and compose associatively — a property that may matter more for hardware than whether the neuron is biologically rich.
Part of the whitebox / MorphoHDL program. Part III of a series with "When faithful isn't useful" (M13.5) and "The interface can dominate the substrate" (M14). Preregistered; the harness was validated by a delayed-copy wiring test in which gradient survival tracked the back-propagation horizon across eight orders of magnitude before any digit was classified.