State-compression ladder · Note M14 · Part X · closer

The small directions still matter

Backprop-free learning, high-dimensional credit, and why variance explained was the wrong compression metric. Three questions, three clean answers — and one that overturns an assumption the whole field leans on.

2026 · whitebox program closer · Parts VI–IX forward-mode credit · credit dimension

By Part VIII the long backward chain was gone: a 1024-step credit path had become independent four-step learners, exact and bit-for-bit, coupled only by boundary notes. One backward operation survived — the tiny local BPTT inside each four-step block. This note removes it, then asks the two questions that follow, and the answers form an arc worth stating up front:

01Removing the last backward pass

Backpropagation runs forward, then sweeps backward to accumulate the gradient. Forward-mode differentiation — the same idea as real-time recurrent learning — carries a sensitivity alongside the state and advances it in the same direction as the dynamics. When the last step is done the gradient is read off. There is no return sweep.

Because forward-mode is exact, the test is unforgiving: the forward-only gradient must equal backprop, and the training runs must land in the same place.

So every level of credit this arc examined turns out not to need a backward pass — long-range (Part VIII), across boundaries (Part VIII), and now local. The backward pass was never necessary for this problem; it is the cheapest exact credit on backprop-friendly hardware, and nothing more.

02Can the credit be compressed?

Forward-mode carries one sensitivity direction per parameter — 82 lanes for this little field. The obvious hope: maybe the field doesn't need 82 exact directions. If a handful of credit channels recovered the same dynamics, forward-only learning would get much cheaper without ever reintroducing a backward pass. Two subspaces bracket the question. A random basis is the lower bound; the top singular vectors of the exact sensitivity matrix — the directions the trajectory actually uses — are the best-possible frozen upper bound. Random failed at once. The real test is the SVD bound, and it is where the surprise lives.

03The small directions still matter

Here is the whole result in one picture: how much sensitivity energy each rank captures, against how much of the oracle's performance it recovers.

Rank-8 holds 97.6% of the sensitivity variance and yet learns a field about 9× worse in free rollout. The tiny singular directions look negligible under an energy norm and are behaviourally decisive.

Gradient agreement says the same from the other side: the top-8 basis aligns with the true credit direction at only cos 0.80 (60% relative error), improving painfully slowly — still 0.96 at rank-64. Both bounds fail, random and optimal. For this field, credit does not live in any low-dimensional subspace; its effective dimension is near full. Compression closes.

04A pattern the programme keeps hitting

This is not merely an implementation footnote. It is the same shape that has recurred throughout the work: large statistical energy does not identify the behaviourally sufficient coordinates.

Each time, a quantity that is easy to measure and concentrates quickly — variance, information, dimension — overstates what the closed-loop behaviour actually requires. The low-energy credit directions are the running lesson: negligible to a norm, essential to the dynamics.

05But it doesn't need compressing

High-dimensional credit would only sting if forward-only credit were expensive. So we asked how much of the cost was fundamental. Almost none. Hand-rolling the batched sensitivity recurrence — Sₖ₋₁ = Sₖ + Δt(JₓSₖ + Jθ), all 82 lanes in one tensor, no generic autodiff — gated bit for bit against backprop:

The exact 82-direction forward credit costs 1.2× backprop when written as a recurrence; the rest of the 7× was generic-autodiff overhead. You cannot compress the credit — and you do not need to.

One caveat, kept explicit because it is easy to oversell. This is not "forward-only learning costs 1.2× in hardware." Vectorization buys latency, not work: the exact all-direction credit is still ~82 lane-forwards of arithmetic, cheap in wall-clock here only because the field is small and the segments already pack the lanes. The ledger has four columns, not one:

For an FPGA or ASIC the attractive part is exactly this: the extra credit lanes are wide but shallow and embarrassingly parallel — many small independent multiply-accumulates, four steps deep. A future GPU result near 1× wall-clock would demonstrate exposed parallelism, not 1× hardware cost.

06What the arc settled

The backward pass was optional: no long-range, no cross-boundary, no local backprop is fundamentally required here. The exact credit is not low-rank compressible — its small directions are behaviourally load-bearing. And exact high-dimensional forward credit still executes cheaply when structured as a batched recurrence: not compressible, but parallelisable enough that compression was unnecessary.

The programme began by suspecting that training a long dynamical trajectory needed something deep, sequential, and backward. It needed none of those — that was a property of how the problem was written. What it does need is honest: credit of genuinely high dimension, because the directions that carry almost no variance carry much of the behaviour. You can restructure a problem until credit is local, forward, and exact. You cannot wish away how many directions the dynamics truly depend on.

07Honest edges

One reduced system (Van der Pol, 82-parameter field), single architecture. The SVD basis is a best-possible upper bound derived from the full oracle — not a deployable cheap method; that circularity is the point, since it makes the negative strong. The 1.2× timing is CPU at one field size, and the forward/backprop work ratio grows with parameter count, so it is a floor on cheapness, not a universal rate. Free-running rollout is load-bearing throughout; variance-explained and gradient cosine are diagnostics, and the gap between them is the finding.

Part of the whitebox / MorphoHDL program. Closer of the multiple-shooting arc (Parts VI–IX). Forward-mode sensitivities (exact, gated against reverse-mode to ~1e-9); random and best-frozen-SVD credit-compression against the forward-only oracle (singular spectrum, gradient agreement, free-run rollout); hand-rolled batched forward-sensitivity recurrence gated bit-for-bit and timed. Reduced Van der Pol multiple-shooting field, explicit Euler surrogate.