Expert-Sharded Swarm Inference: The Design

86% of a 122B MoE is 12,544 independent 5.3 MB experts. Slice the model at that grain and a phone can carry a real share of frontier inference.

The expert index is the outermost dimension of every MoE tensor, so each expert is a contiguous, quant-block-aligned slab. An expert-sliced mini-GGUF is a clean byte-range copy — no dequantization, no re-packing.

• Backbone stage (strong node): attention + KV cache, all norms, the router, the shared expert, and the residual combine — the entire dense path. It also keeps all experts resident as a fallback replica (RAM offload), which gives the swarm churn tolerance.
• Expert worker (a phone): not a transformer. No attention, no KV, no sampler — a pure function (hidden, local_ids) → out made of three mat-muls, holding only its own expert slice. It fits any budget down to a 4 GB phone.

cur   = ffn_norm(x)                       # backbone
ids,p = top_k(softmax(cur @ router), 8)   # backbone — authoritative
── dispatch selected experts to owner nodes ──
send  (cur rows, local_ids)  →  worker    # ~6 KB per decode step
recv  expert_out             ←  worker
x = x + combine(p, partials) + shared(cur)  # backbone — numerically exact

Because the router runs exactly once on the backbone, each selected expert is computed exactly once by whichever node owns it. There is no approximation — the sharding only moves where the mat-muls happen.

Kvasir already runs an autonomous scarcity market for layer shards, verified on real devices: a NAT-bound phone polls the demand map, self-enrolls into the highest-reward segment, partially downloads only that window, loads it on its Adreno GPU, and completes ring inference — earning contribution rewards. Expert sharding reuses all of it — coverage map, max-reward self-enroll, partial download, per-node rewards — changing only the coverage unit from *layer ranges* to *(layer, expert-range)*.

• Reward-driven partial-weight download: conventional RPC/TP/PP setups ship the full checkpoint to every rank and a scheduler dictates placement. In Kvasir a node downloads only the slice it will compute, and picks that slice itself, by reward — a 254 MB stage mini-GGUF versus the 77.6 GB full model. This is how a 4 GB phone joins a model far bigger than itself.
• 443 relay data plane: Cloudflare's 80/443-only edge plus carrier NAT means no direct dial in either direction. A per-edge WebSocket bridge with a 1-byte role preamble lets both sides dial outbound (the phone opens zero inbound ports). Landing it meant fixing three real bugs — build-fingerprint agreement, node-token download auth, and a Kotlin Int.ushr frame-length bug that silently corrupted every frame ≥ 64 KiB (ushr uses only the low 5 bits of its shift; len ushr 56 became len ushr 24) — fixed by moving to Long shifts.

Decode is 49 serial layers, and a cross-internet round trip per layer costs 2.5–10 s per token. So the swarm's contest is serving models nobody can host alone, measured in aggregate throughput: batch dispatch amortizes RTT, the backbone keeps a hot-expert cache, and requests route to near replicas. The low-latency path stays with the pipeline ring.

• M0 — backbone expert RAM offload: run 122B on one coordinator, no graph surgery.
• M1 — single-host expert-parallel proof: expert-sliced mini-GGUF + worker runtime + dispatch, logits exactly matching monolithic.
• M2 — LAN + NAT phone workers computing real 122B experts through the 443 relay.
• M3 — expert-grain coverage market with replicas and churn fallback.
• M4 — throughput: batched dispatch + hot-expert cache, tokens/s scaling with worker count.