to silicon

A trained language model became a placed-and-routed FPGA netlist, and every link in the chain is verified — most of them bit-exactly. This page is the record of that chain: what was built, what was measured, what the honest throughput predictions are, and where the ceilings actually sit.

the chain

Each stage below exists as running code, and each handoff was checked against the stage before it — no stage trusts the previous one's claims.

why this was a transcription, not a port

What Morpho's compiler emits is already a hardware netlist: k-input LUTs with explicit integer truth tables, registers, two constants, wires, one implicit clock. An FPGA is a fabric of exactly those primitives, so the exporter (tiny_morpho_hw.py) maps one-to-one and adds nothing:

GATE(lut, args)      →  a physical LUT with that exact truth table
REG(init)            →  a flip-flop  (.latch in BLIF, always @(posedge clk) in Verilog)
synchronous commit   →  the clock edge

The synthesizable subset is the REG-only discipline — every feedback loop broken by a register. The FORWARD/TIE fixed-point corner (the SR latch) is deliberately rejected: it sits outside what synchronous tools guarantee anything about.

verification, three layers

1 — the text is the circuit. An independent simulator re-parses the emitted BLIF text (never the compiler's data structures) and re-runs it against the compiled circuit: exhaustive for the 2^(−x) unit (all 65,536 Q8.8 inputs), thousands of random cases for the dividers, 120 channels × 32 ticks for the full wkv cell. All bit-exact.

2 — the two formats agree. yosys formally proves the emitted Verilog ≡ the emitted BLIF (registers included) for the three smaller units. For the wkv cell the SAT solver hits the classic exponential case — equivalence over multiplier cones — so that check is time-bounded and reported as not proven within budget, never as success; the text round-trip covers its behavior.

3 — a real tool placed and timed it. f_max below is nextpnr's static timing on a routed design, not our logic_depth estimate.

measured results — iCE40 HX8K

The serial-divider row is the article's space→time rotation measured by a real place-and-route tool: threading the remainder through a register instead of through space turns a deep, slow circuit into a tiny, fast one.

The wkv row is the headline: the novel part of RWKV — the recurrence that doesn't exist in any commodity accelerator — fits in 73% of a $10 FPGA. Its 9.8 MHz is the honest cost of doing exp → multiply → divide in one combinational pass; pipelining that path is a design knob, not a research question. Its state is three 16-bit registers per channel: the entire model's recurrent memory is 6,912 bits.

so what does it generate — predicted tokens per second

The per-token invoice is fixed by the model's own manifest: 13.1 million int8 multiply-accumulates — and since int8 is one byte per weight, batch-1 generation must also stream 13.1 MB of weights per token. The wkv work is 2,304 channel-steps (6 × 384). Three ceilings follow, and the lowest one wins:

bandwidth ceiling   tok/s = bytes/s ÷ 13.1 MB      rule of thumb: MB/s ÷ 13
compute ceiling     tok/s = MAC/s   ÷ 13.1 M
wkv ceiling         tok/s = cells × f_max ÷ 2,304   (measured: 9.8 MHz ⇒ 4,250/cell)

The shape of this table is the finding. The novel silicon never binds: one wkv cell at its measured worst-case clock already covers 4,250 tokens/s, two orders of magnitude above any realistic build's bottleneck — which is the same weight-bandwidth wall every GPU and CPU hits. RWKV's contribution to the hardware problem is that everything outside the matmuls became three registers and a small fixed-point pipeline; the matmuls themselves are commodity MACs and a memory system, solved problems bought by the gigabyte.

honest edges

These predictions are ceilings from accounting, not measurements — a real build loses some fraction to DRAM row misses and scheduling, so treat the table as "within 2×" until a board runs. Prefill is a different, faster regime: the matmuls batch over the whole prompt while only the cheap wkv scan stays sequential. Timing closure numbers are nextpnr's default corner. Large ROMs and the matmuls should map to BRAM/DSP macros rather than gate-level LUTs — an exporter feature, not a language problem. And the last two steps to a blinking dev board are icepack (the routed .asc already exists) and owning the $10 part.

Everything here is reproducible from the repository: python3 examples/rwkv/wkv_cell.py re-proves the bit-exactness, python3 examples/hardware/export_units.py re-runs the export, round-trips, equivalence checks, and place-and-route (brew install yosys nextpnr-ice40 icestorm).

AI-generated research. The experiments, code and write-ups on this site were produced by AI agents (Claude and Codex) working semi-autonomously, directed at a high level by a human. Nothing here is peer reviewed and none of it is affiliated with an academic institution. Measurements are reported as taken — including the nulls, the caveats and the corrections — but the work has had no external review.