The 200G Question
Our swarm hubs can already link at 200 Gb/s with parts on the shelf — one is built into the GB10. Here is what a fat pipe buys a distributed MoE, and the one thing it can't.
The premise: WAN decode is RTT-bound, not bandwidth-bound — our comm roadmap showed bytes are the easy part. So what actually changes when hubs get 200 Gb/s links? Almost everything about *capacity*, and almost nothing about *latency*.
The hardware is not futuristic — one ships inside our GB10 worker
The GB10 Grace Blackwell that computes our 122B experts carries an NVIDIA ConnectX-7 with two 200 GbE QSFP ports on board. Two of these machines direct-connect with a single ~
mlx5 driver stack that runs these NICs in x86 datacenters runs them on aarch64, which is exactly what the GB10 is.
The fine print: the GB10 feeds its ConnectX-7 through two PCIe Gen5 x4 links in multi-host mode. Measured full speed (~185–190 Gb/s) requires RoCE (RDMA) and a correctly mapped topology — naive TCP over a mis-mapped path lands at ~95 Gb/s or worse. Fat pipes are bought with configuration, not just cables.
200G is a catalog item at every reach
| reach | part | form factor |
|---|---|---|
| rack (0.5–3 m) | QSFP56 DAC copper | cable, ~ 00 |
| room (~30 m) | AOC active optical | cable |
| campus (2–10 km) | 200G FR4 / LR4 optics | QSFP56 module |
| metro (~40 km) | 200G ER4 optics | QSFP56 module |
| region (~120 km) | 400G ZR+ coherent, run at 200G line rate | QSFP-DD module |
| long-haul (100s of km) | carrier 200G wavelength / DWDM line system | leased service |
In the WAN, the *cable* is just standard single-mode fiber — speed-neutral glass that already spans every city. The speed lives in the pluggable optics at each end, and OpenZR+ made 200G-over-120 km a module you plug into a switch, not a telecom project. Beyond that, you lease a wavelength.
Every bandwidth term in the swarm vanishes
- A dispatch payload (~110 KB/token/layer today, ~10 KB after the wire roadmap) serializes in microseconds — payload size stops being a design constraint at all.
- An expert slice ships in ~32 ms (794 MB, theoretical) and a whole 122B model syncs in ~3 s — coverage-market rebalancing and new-hub onboarding become near-instant.
- Long-context prefill — the one genuinely bandwidth-heavy phase — moves at wire speed, so first-token time on 100K-token prompts becomes backbone-compute-bound.
- Batched dispatch scales without a wire ceiling: expert-pool traffic aggregated across many user streams is exactly the bandwidth-heavy, latency-tolerant load a fat pipe absorbs. This is what makes a multi-backbone federation — several hubs, each holding KV for its own users, sharing one expert pool — practical.
Light does not hurry
Fiber carries light at ~5 µs/km, and no amount of bandwidth changes that. A 13 ms round trip is 13 ms at 200 Gb/s. Autoregressive decode pays that round trip per sharded layer, per token — which is why speculative decoding (k tokens per round trip) and shared-expert overlap (compute while the dispatch is in flight) stay essential even between hubs joined by the fattest pipe on the market. Bandwidth buys throughput; only round-trip discipline buys latency.
So the architecture settles into two tiers. A hub tier — backbones and hot experts joined by 200G-class links, where capacity is effectively unbounded — and an edge tier — phones and small devices on the 443 relay, holding the long tail of experts the scarcity market assigns them. The fat pipe makes the first tier feel like one machine; the relay keeps the second tier open to anyone. Neither replaces the other: that split *is* the design.