From Blueprint to Hardware: What's Verified, and the Keystone

A recap of the verification campaign — design through M2-core proved on a real 122B — and the one integration piece that unlocks the rest.

Over the past weeks, the hard, novel pieces of the expert-sharded swarm were proven one by one on a real Qwen3.5-122B — not simulated, not toy-sized. Here is the verification trail so far, and the single keystone that remained.

A verification trail of stamped checkpoints ending at a keystone being placed
The trail · all verified on the real 122B

What has landed so far

  • Design (5 revisions since the blueprint) — EP architecture, partial-weight autonomous participation, the relay, worker kernel spec, and cross-backend numerical equivalence codified as core technology. Router authority written down as the coherence invariant.
  • M0 — backbone expert RAM offload (planner verified): 122B is *feasible* on a single 64 GB coordinator — 10 expert layers offloaded to RAM, VRAM 62.6 GiB / RAM 14.2 GiB, wired via --override-tensor.
  • M1 — expert-slice data path: per-expert mini-GGUF slicing (ne[2] slabs, byte copy without dequant) + the /expert-shard download endpoint.
  • M1 — numerical oracle: dispatch + combine == monolithic with max|Δ| = 3.6e-12 on real layer-0 experts — sharding is an exact regrouping of the same weighted sum.
  • M1 — C++ worker on hardware: linkcpp-expert-worker (pure ggml/gguf) built and run on ROCm, cosine 0.99995 vs the oracle; router → two C++ workers → combine matches monolithic at cosine 0.9997–0.9999.
  • M2 core — a phone computes real 122B experts: Android cross-build, run on an SM-S938N, cosine 0.99992 vs the oracle.
  • Numerics — 3-backend equivalence matrix: the same 122B computation on ROCm × phone ARM CPU × numpy — ROCm↔phone cosine 0.99990, ROCm↔numpy 0.99996, phone↔numpy 0.99992. All equivalent, none bit-identical.
The keystone

Backbone dispatch, integrated into live decode

Every component — slices, workers, dispatch/combine logic, numerical equivalence, phone compute — was device-verified. What remained was wiring them inside a real inference engine decode: a build_moe_ffn hook that dispatches experts to their owner nodes mid-graph. It required modifying the pinned inference engine submodule and several build-verify cycles. Once this keystone stands, M2 relay integration, the M3 expert coverage market, and M4 batched throughput open in sequence — all of them depend on this dispatch.

The keystone has since landed: the follow-up posts on M2, M3, M4 and the live phone demo are the results of exactly this integration.