Phase 5 — Distributed Expert Dispatch (M2)

A live 122B decode hands one layer's expert compute to a separate worker process over TCP — and predicts exactly the same token.

Verified · real 122B, two processes

Backbone decode → TCP → worker → experts → same token

MATCH
argmax OFF == ON (11751)
0.99869
logit cosine
0
transport loss (byte-identical)
2
processes (backbone + worker)

The expert worker serves the layer-0 slice as a separate process (ROCm), and the 122B backbone's build_moe_ffn dispatch callback ships (cur, sel) over TCP and receives the expert outputs. The logit cosine is exactly the in-process value (0.99868775) — the transport is lossless. Expert-parallel swarm compute works across a process boundary.

Backbone and worker rooms joined by one TCP pipe, sealed with an argmax MATCH stamp

One long-lived TCP connection — the same stream the ring/443 relay can tunnel.

# worker: serving as a separate process
linkcpp-expert-worker --serve 52700 --model L0_all.gguf --layer 0 --n-embd 3072
# backbone: build_moe_ffn callback dispatches to the worker
linkcpp-moe-verify 122B.gguf ... --dispatch-port 52700
  → protocol: [n_used, n_tokens] + cur + sel  →  experts
Done

The distributed dispatch pipeline

  • --serve mode: load the slice, listen on TCP, answer (n_used, n_tokens, cur, sel) → experts.
  • --dispatch-port: the backbone callback sends/receives over TCP to a separate worker, replacing in-process compute.
  • Measured on a live 122B decode with layer-0 dispatched out-of-process → argmax MATCH, cosine 0.99869 (= in-process, lossless).
  • M2 core (earlier): the phone's ARM computed real 122B experts at cosine 0.99992 (Android cross-build).

Next from here: tunneling the same TCP stream through the 443 relay to workers on other machines and phones (the transport was already proven in the ring work), then the M3 coverage market and M4 batched throughput.