Whitebox LM · M13 · preprint
How much of learning a dynamical neuron can be decoupled from the neuron itself?
We ask how many dynamical degrees of freedom a neuron's computation actually requires, and — more sharply — which parts of learning a self-coupled dynamical system can be parallelised and which cannot. Using a Hodgkin–Huxley teacher whose four state variables are known, we build an instrument (a compression ladder) and drive it through eleven preregistered rounds, an external code audit, and a single-shot held-out evaluation.
Two results survive. (1) A compact primitive: a frozen, parallel-trained vector field plus a ~34-parameter, single-scalar bounded correction reaches ≈0.88 held-out spike-timing F1 with the correct anodal-break rebound and near-perfect firing-rate curve. (2) A boundary: local dynamics, state representation, and even own-trajectory generation all parallelise — but high-fidelity closed-loop learning was reached only by long-horizon sequential credit assignment. Freshness, short-block coupling, and every offline distribution/label/objective correction we tried fell short.
The neuron is usually the cheap part of a network — a weight and a nonlinearity. This study starts from the opposite premise: that a neuron is a small stateful dynamical system, and asks what that state is for, how little of it is needed, and — the question that turned out to matter most — how much of the difficulty of learning it is irreducibly sequential.
Working against a known-dimensional teacher lets every claim be an instrument reading rather than an assertion. What follows is organised as the eight questions the programme actually answered, in order.
We built a collision diagnostic: two representations that map to nearby points must demand consistent futures, or the representation is insufficient. Applied to learned encoders of the HH state, sufficiency drops sharply and then plateaus — a knee at roughly three effective coordinates, consistent with the classical fast-m reduction (V, h, n) rather than the full four. The instrument measures a minimal realisation for a model class under an input distribution, not a canonical dimension — an important scoping that recurs throughout.
Instantaneous voltage is ambiguous (collision ratio 0.35). A rolling 12 ms window of five voltage samples reaches the same sufficiency floor as the full-state encoders (0.14). Informationally, recent history substitutes for the hidden gating variables — a measured Takens-style result on the exact data.
Yes, and cheaply. Training a vector field on independent (state, input) → dV/dt samples is roughly 15× faster than sequential rollout training and reproduces the firing-rate curve and the anodal-break rebound. An integrator-isolation control (RK4 vs Euler on identical weights) came back null: discretisation is not the bottleneck.
The parallel-trained field sits at ≈0.74 spike F1 despite excellent waveform statistics. Purpose-built metrics found the cause, and it was none of the obvious candidates. Spike-miss F1 is flat across the rollout — not accumulating drift. Misses are episodic: P(miss | previous miss) = 0.69 against a base rate of 0.29. Episodes are entered rarely, then healed by the input in ≈5.4 ms. Initiation is a slow inter-spike under-excitability drift (n rising, h falling) visible ~5 ms before the missed spike. The plateau is an event-decision problem, not a stability failure.
Freeze the good field; add a small recurrent correction, hard-bounded to the empirically located drift/decision voltage band so it cannot rewrite the physics. It crosses the original F1>0.9 gate that every prior model failed. The correction is inspectable: its magnitude is near-zero everywhere except the drift and decision bands — the credit state doing exactly the job the causal anatomy predicted.
| model | corr. params | state/neuron | held-out F1 (s0/s1) | f-I (Hz) | rebound |
|---|---|---|---|---|---|
| trust-hybrid, kc=8 | 396 | 8 | 0.879 / 0.888 | 0.5 / 0.3 | ✓ / ✓ |
| trust-hybrid, kc=1 | 32 | 1 scalar | 0.861 / 0.836 | 0.6 / 9.9 | ✓ / ✓ |
| observable parallel baseline | — | — | ≈0.264 | — | — |
| sequential GRU (long-horizon) | 930 | 8 | 0.869 | 14.5 | ✓ |
No. Compressing the corrector from 396 parameters to 32 parameters and one scalar of runtime state costs almost nothing. And in complete coordinates, a matched-parameter static correction performs the same as a recurrent one — so recurrence, at matched size, adds nothing when the coordinates already expose the state.
Exactly when the coordinates hide it. The sharpest contrast in the study: given the same 12 ms voltage history, a static map fails utterly — 0.004 F1 even with oracle-perfect history — while a recurrent model reaches 0.87+. Identical information, opposite outcomes. Internal state is a computational resource for reconstructing hidden state under partial observation, not a universal enhancer. A selective (Mamba-style) affine accumulator matched the GRU at a fraction of the training cost — the right machinery is cheap and parallel-trainable.
Everything else was exonerated by preregistered control: state amount, state form, capacity, field-error magnitude, error placement, label alignment, and the state-shaping objective (a JEPA-style latent-future target changed nothing). The observable ceiling is ≈0.264. What remains is the coupling between a model's parameters and the trajectory distribution those parameters induce.
We priced it directly. Offline training on the model's own distribution — at any mixture up to 100% — collapsed. A block-coupling experiment (in the style of test-time training) then separated the two candidate resources with a built-in invariant: at block size 1, where no through-time credit exists, the detached and backprop-through-time arms are byte-identical, validating the harness. Above it, neither fresh-state resynchronisation nor short-block through-time credit crossed the gap. Only long-horizon sequential training ever did.
The decomposition has a clean deployment reading. The dynamics are trained once, paying the one-time sequential cost. Thereafter the primitive freezes: a shared ~34-parameter law plus one scalar of state per neuron. Downstream networks train connectivity and readouts around the frozen cell with no coupling problem at all, because a frozen field's rollout distribution no longer moves. Bounded per-population adaptation stays parallel; only rewriting the core dynamics reawakens the sequential cost. Coupled training is a cost you pay to mint the primitive — not one you repay on every use.
An independent code-path review found indexing (an off-by-one in the flagship target), a chunk-boundary regularisation error, current-inconsistent phase labels, and a data-budget confound in the coupling sweep. All were corrected and the affected experiments rerun. The corrected runs preserved every headline conclusion — and tightened one: the single-scalar corrector improved with correct targets. The observable baseline was corrected upward (≈0.20 → ≈0.264). A closed-form scan implementation was unit-tested against an explicit recurrence to 1×10⁻⁷ in value and gradient. Final performance was then measured once on a freshly generated locked corpus. Repeatedly-scored figures are labelled development metrics; the held-out numbers above are the reported test performance.
The primitive is proven; the next question is whether richer dynamics buy computation per unit hardware. That becomes a measurement — Information Processing Capacity (linear and nonlinear orders) per state-bit, per gate, per picojoule — comparing the frozen primitive against LIF, AdEx, and a matched timescale-spectrum control. Beyond it: a four-neuron re-taskability test (fixed weights, changing only a context vector) as the bridge toward synchronisation-as-computation. Connectivity cost, which scales roughly quadratically, is measured separately from neuron cost — the real hardware argument may be that a richer neuron needs fewer neurons and therefore far fewer connections.
A whitebox LM research note. Full lab notebook — including preregistrations, failed recipes, published corrections, and every intermediate result in chronological order — plus a clean reproducibility path (repro/) at github.com/PedalCore/state-compression-ladder. Eleven development rounds and this synthesis were shaped throughout by an external reviewer's critiques; the audit above is theirs. Part of soundlark.studio — see also the white-box lab and thinking in ticks.
AI-generated research. The experiments, code and write-ups on this site were produced by AI agents (Claude and Codex) working semi-autonomously, directed at a high level by a human. Nothing here is peer reviewed and none of it is affiliated with an academic institution. Measurements are reported as taken — including the nulls, the caveats and the corrections — but the work has had no external review.