soundlark research notes

Developmental spiking neural networks as interactive music — what we built, what we measured, and what honestly didn't work.

The idea

MorphoHDL grows circuits by recursive structural rewriting. We asked: what if that recursion grew a spiking neural network instead — and the network's own activity decided which parts of its structure grow, survive and are pruned? And what if everything it did was audible?

The result is a browser laboratory. A compact recursive grammar expands into a recurrent leaky integrate-and-fire network. Pitch is anatomy: developmental depth sets the register, structural position sets the scale degree — so structural events are musical events. Development runs on survival energy: neurons that fire (or are played) live, neglected anatomy starves and is pruned. Everything is deterministic per seed and covered by a headless test suite (48 tests) that drives the exact code the browser runs.

The three instruments

The Lab — the organism plays itself: rhythmic input drive, stochastic walkers as melodic voices, rare "modulator" neurons that nudge the key around an interactive circle of fifths, region divisions opening new registers, microtonal tunings, rubato. The Duet — the human replaces the metronome: notes (MIDI, pads, keyboard) are encoded as spike bursts into a tonotopic sensory layer; the organism answers in the gaps you leave, at the tempo you asked at. Attention — the duet plus attention-modulated spiking (below), left as a separate page so the plain duet stays an experimental baseline.

What we measured

1. Answering at all is an architecture problem

A recurrent LIF net this size holds activity ~100 ms after input stops: it cannot "answer in the gap" — it goes silent, or occasionally seizes (300-note runaway bursts). The fix is architectural: when your phrase ends, walkers are teleported onto the anatomy your call activated, and their weighted traversal is the answer. Result: 100% of calls answered, immediate call/response relatedness ≈ 0.5 (degree-histogram cosine), because the seeding is tonotopic by construction.

2. What does it learn? Usage — not sequences

Motif-completion probes with controls: train a 4-note motif 50×, cue with its first two notes, measure whether responses predict the continuation.

A specialization control (train idiom A, probe idiom A vs unseen idiom B over a session) likewise came back flat: the organism mirrors; its constant A-over-B advantage exists before training and is architectural. These nulls are the most useful results we have — they say precisely where the substrate's ceiling is.

3. Attention: the MA-SNN result reproduced musically

Adapting Attention Spiking Neural Networks (gradient-free): leaf regions are the "channels"; each region's synaptic input is gated by how well its pitch material matches recently heard notes. Ablated on 60-exchange sessions, 4 seeds, all arms with STDP + development:

Temporal mixing — inspired by SpikeVoice's Spiking Temporal-Sequential Attention (ACL 2024) — attends over context depth (three timescales of heard material, sharpness-weighted) before attending over structure. Under a static-idiom protocol it ties with plain suppression; its expected advantage (adapting when the player's material changes) is untested — the fair test is an idiom-switch session, queued as future work.

4. Attention as morphogen

Because the attention "channels" are also the developmental units, attention can feed survival: strongly attended regions receive energy, so what the player attends to is what develops. Ablation over 120-exchange sessions: idiom coverage of the anatomy rises in both arms (0.56 → 0.64 — activity-driven development is already a morphogen), with the attention trickle adding a modest sharpening (Δcoverage +0.077 vs +0.062; late relatedness 0.596 vs 0.566) and — striking — organisms that stay ~50% larger, because attention feeds anatomy that pruning would otherwise starve.

5. Style capture: Beethoven in, Beethoven-ish out

Train on the Ode to Joy theme (30 passes), then probe with simple novel cues and compare responses to the theme's distributions, against an untrained twin of the same genotype:

6. Sideline: language

The same substrate pointed at tiny Shakespeare — reservoir prediction, a Forward-Forward readout, and error-driven growth toward adequate capacity — has its own page: language notes. Short version: nowhere near transformers (expected), but developmental exposure measurably improves the reservoir, replicated.

What the Morpho layer adds — and what it costs

Every attention paper we borrowed from uses a fixed architecture. Here the attention channels are the developmental units, which makes attention-shapes-structure a one-line integration rather than a research program. The cost is the mirror image: no gradient path, tiny graphs, topology that changes under you — chasing learned-attention results at scale belongs in a separate fixed-topology effort, not this substrate.

Living with an organism

Organisms are now persistent: full deterministic snapshots (every membrane potential, in-flight spike, plasticity trace, RNG stream) — a restored organism continues spike-for-spike identically, verified by test. Save/load lives in the browser; export and import move organisms between people as files. And any MIDI file can be the player: train from midi replays a score into the sensory layer at whatever sim speed you like, then you jam with whatever it became.

Where this goes

Rhythm learning (walker IOI vocabularies), idiom-switch tests for temporal attention, reward shaping through play (the reinforce button already drives R-STDP eligibility), overnight MIDI-corpus training on persistent organisms, and porting the validated mechanisms — walker answers, suppress attention, structural pitch, energy survival — to the native C++/JUCE instrument this research track runs alongside.

Everything here is reproducible: cd snn && npm test (48 tests) · npm run experiment:learning|dialogue|attention|specialization|morphogen|rstdp|style — each prints the tables above from scratch. Built on MorphoHDL (Apache 2.0). References: Yao et al., Attention Spiking Neural Networks; Wang et al., SpikeVoice; Izhikevich, Solving the distal reward problem (R-STDP); Dynamic Spiking GNNs; SGNNBench.

AI-generated research. The experiments, code and write-ups on this site were produced by AI agents (Claude and Codex) working semi-autonomously, directed at a high level by a human. Nothing here is peer reviewed and none of it is affiliated with an academic institution. Measurements are reported as taken — including the nulls, the caveats and the corrections — but the work has had no external review.