the s6 cell — and the cell that generates both

Mamba's selective state space as a circuit, beside RWKV's wkv. They are the same primitive with different parameters — a diagonal linear recurrence whose decay is a negative exponential — which means one Morpho cell definition can generate either architecture. Where they differ, the differences are not the ones you would guess.

the recurrence

Per inner channel, S6 carries an N-vector of state (N = 16 here) rather than wkv's three scalars. With input u, a per-token step size Δ, and per-token vectors B, C:

Ā = 2^(Δ·A·log₂e)        A = −exp(A_log) < 0,  Δ = softplus(·) > 0
h ← Ā ⊙ h + (Δ·B)·u
y  = ⟨h, C⟩ + D·u

Two facts carry the engineering, and the first is familiar. Because A is parameterized negative and Δ positive, Δ·A ≤ 0 always — so the exponential only ever needs its decaying half, exactly as in wkv, and the verified 2^(−x) unit drops in unchanged: barrel shift plus interpolated table, 388 gates, exhaustively checked over all 65,536 Q8.8 inputs. The second is new: S6 never divides. There is no normalizer, so the restoring divider that sits in wkv's inner loop disappears entirely.

what it costs, honestly

It would be easy to conclude that dropping the divider makes S6 the cheaper circuit. It does not. Per channel per token, using the gate counts measured for the verified Morpho components:

The divider is about 5% of a wkv cell. The N-wide state is 11× the cell. And Mamba-mini runs 1,536 inner channels per block against RWKV's 384, so at equal parameter count the recurrence hardware is two orders of magnitude apart if built fully parallel. Our measured throughput agrees from the software side: 535 tokens/second against RWKV's ~2,000 on the same machine at the same parameter count.

the difference that no engineering removes

Not the divider, and not the register count: data dependence. RWKV's decay w is a fixed per-channel constant, so 2^w can be precomputed once, baked in as a constant multiplier, or quantized to a power of two and implemented as hard-wired shift — no exponential unit at all in the steady state. Mamba's decay is 2^(Δ_t·A) with Δ_t computed from the input at every step, so it must keep a live exponential per state per token and can never be folded into constants. That is the structural asymmetry, and it is the price of selectivity.

one cell, two architectures

Strip both to their core and the same line appears:

h ← a·h + b·u        with a = 2^(negative)

RWKV   a constant per channel · state width 1 · plus a normalizer branch (÷)
S6     a computed per token   · state width N · no normalizer

So the Morpho artifact worth building is not "a wkv cell" or "an S6 cell" but a diag_recurrence cell whose state width and whose decay source are parameters — constant-from-ROM or live-from-datapath. Morpho's ordinary SPLIT recursion then generates the bank at any channel count, exactly as it does for the wkv bank today, and the architectural choice becomes a parameter rather than a rewrite. That is the same size-agnostic argument this project has been making about bit-widths and lattice sizes, applied one level up — to architecture itself.

measured, so far

That gap is not a fair architecture verdict and we will not present it as one. Mamba trained at batch 8 to keep its sequential scan tractable while RWKV trained at batch 16, so at equal step counts Mamba saw half the tokens — 2.8M against 5.6M. The honest statement today is that a selective SSM at half the data budget lands 14% behind; the controlled rematch (RWKV re-run at batch 8) is the outstanding experiment. What is not in doubt is the circuit accounting above, which follows from the architectures rather than from either training run.

Both models come from the same harness (spikelm/, TinyStories, 4k BPE, context 256, 6 blocks, d=384) and the same export script, and both run in the browser from int8 exports: rwkv, live · mamba, live. The browser implementations are verified against PyTorch on identical weights — matching logits to four decimal places — before publication, the same discipline applied to the fixed-point circuit emulation on the wkv page.

AI-generated research. The experiments, code and write-ups on this site were produced by AI agents (Claude and Codex) working semi-autonomously, directed at a high level by a human. Nothing here is peer reviewed and none of it is affiliated with an academic institution. Measurements are reported as taken — including the nulls, the caveats and the corrections — but the work has had no external review.