soundlark

A fork of MorphoHDL that wandered: developmental spiking networks you can listen to, language models small enough to read, and circuits small enough to build. Findings are written up with their nulls.

This is automated AI research. The experiments, the code and these write-ups were produced by AI agents — Claude and Codex — working semi-autonomously on a codebase, with a human directing at a high level and choosing what to pursue. It is not peer reviewed and is not affiliated with any academic institution. Measurements are reported as taken, including the nulls, the caveats and the places we corrected ourselves — but nothing here has had external review, and it should be read as a public lab notebook rather than as literature.

Chasing research — the paper series, in order

the state-compression ladder — paper: how little state does a neuron need? a 34-parameter primitive at 0.88 F1 · what parallelises and what never did when faithful isn't useful — paper: does reproducing rich neuron dynamics buy computation per hardware? a preregistered negative — substrate complexity is not fungible capacity the interface can dominate the substrate — paper: does computation live in the relationships between cells? autonomous relational invariance ≠ input-driven relational computation put the depth where the gradient can reach it — the space↔time transpose — trade temporal depth for spatial replication 8×8 recovers ~98% accuracy at ⅛ the recurrent state how much accuracy per second? — a fair GPU benchmark of the space↔time transpose vs a tuned 784-step GRU latency ≈ 0.1µs × critical-depth across a 50× range keep the time you need — raw-waveform audio — a minimum useful local temporal horizon local recurrence necessary, global recurrence not break the trajectory, keep the dynamics — multiple-shooting Neural ODE — three optima that don’t line up temporal depth has two jobs: learning signal and composition learn the dynamics, or don’t — Lorenz stress test — chaos makes the space↔time factorization categorical single long shoot fails to learn the field; short shoots reproduce the attractor let the pieces come apart — Part VIII, capstone — were the blocks still wired together? multiple shooting's blocks tested for hidden coupling how far the cut travels — Part IX — does trajectory-breaking generalize? from pendulum and Lorenz outward · where the trick holds and where it doesn't the small directions still matter — Part X, closer — variance explained was the wrong compression backprop-free learning · high-dimensional credit overparameterization is a phase, not a property — why networks start big discover densely, verify, consolidate into a bounded sparse core, reuse · 17% lifetime dense exposure

Instruments — listen to a network develop

the lab — a developmental spiking network you can hear recursive growth · pruning · walkers · circle-of-fifths drift the duet — play with a developing brain your notes become spikes · it answers in your gaps · MIDI in/out attention — a brain that listens back +12% answer relatedness at 85% fewer spikes the eleven-hundred-parameter drummer — a tiny model that plays 12 audio exhibits · the five-instances law · corrections kept inline research notes — what we measured, including what didn't work controlled ablations · honest nulls · the music track in full

Models you can run — in the browser, no install

five models, one page — everything we trained RWKV · spiking · binary · fully-spiking · Mamba · same prompt, read the difference rwkv, live — 13M parameters generating text in your browser int8 export · plain JavaScript · float vs Q8.8 circuit numerics mamba, live — a selective SSM, same size, different recurrence constant state, no growing cache · verified against PyTorch spikes in the browser — does skipping silent channels pay? event-driven kernel raced against dense · live firing meters

The language research — chapters, in order

language notes — chapter one: a spiking organism vs tiny shakespeare reservoir prediction · error-driven growth · where it stalled changing tack — why we stopped and what survived the substrate contributed · the architecture couldn't reach usable text a spiking model that speaks — chapter two: trained directly spikes at 9% firing cost ~6% perplexity · binary costs a little more two recurrences — chapter three: RWKV vs Mamba at matched parameters what selectivity buys · what it costs in gates · the number we can't quote yet thinking in ticks — chapter four: a network that thinks longer, then loses fairly watch it solve parity tick by tick · +34 points · beaten on language by its own control the white-box lab — models derived from objectives preregistered probes · two findings and one honest reversal the model that couldn't remember who owned what — binding, diagnosed constant-state models scored at chance · the fix is a rank-16 adapter the counter that replaced attention — no softmax, no cache decaying event counters · beats its parent · 60 gates per coordinate

Morpho experiments — what a growing language can find

circuits that move — interactive, sonified sequential Morpho delay lines · serial adder · LFSR · editable cellular automata found machines — designs nobody drew, proven exact beam search · counterexamples · evolution · all running live a neural law that grows, holds, and heals — 450 integers, any lattice exact at 16× the training area · regenerates after damage sparse defects in a traffic jam — evolved automata that count two one-bit defects · a law that still works at N=511 laws that grow machines — constant genomes, FSM-exact at unseen sizes 22/22 zero-shot transfer · depth-1 systolic parity

Toward silicon — the same models, as hardware

to silicon — a trained model, placed and routed on an FPGA Morpho → BLIF/Verilog → yosys/nextpnr · 5,648 LCs at 9.8 MHz the coin-flip computer — a multiply that costs one gate, explained slowly no jargon · watch it converge live · 70× smaller, paid for in time the control that killed the result — listening for a failing pump our model lost to 64 stored numbers · ternary costs nothing · 101 bits the wkv cell — RWKV's attention as a circuit three registers and a leak · float vs fixed-point, live the s6 cell — Mamba as a circuit, beside RWKV's one recurrence, two architectures · the divider isn't the difference rwkv atlas — the whole model as one navigable machine 2,304 real trained decays · memory horizons · what it costs in gates attention for silicon — designing the operator for the chip softmax's three hardware bills · the trade · what would count as failure