The swarm that grows under load
A giant model that borrows help only when it needs it — the MoE swarm now scales itself to traffic: tight and fast when quiet, wide and parallel when busy.
Kvasir serves models far larger than any single machine holds — a 122B-parameter Mixture-of-Experts model runs across a coordinator plus a swarm of workers: GPUs on the LAN, GPUs across a 200 Gb/s link, even phones that dial in over the internet. Because an MoE model routes each token to only a handful of its experts, most of the weights sit idle at any moment, and those idle experts can live off the main node — on whatever hardware volunteered to hold them.
The new part is that the swarm now scales itself to the load.
Tight when quiet, wide when busy
When traffic is light, the coordinator serves everything on its own GPU — the fastest path per token, no network hops. When requests start piling up and its inference slots saturate, two things happen automatically:
- It re-engages the workers it already has. The coordinator watches its own queue. Under saturation it keeps streaming routed-expert work out to proven workers — ones that have actually served before — trading a little per-token latency for a lot more total throughput. A worker that merely connected but never computed is never trusted with load; a brand-new worker still gets a fair first try.
- The hub recruits new ones. The control hub notices the same saturation and raises the "demand" for that model's experts. Idle nodes — a phone in someone's pocket, a spare GPU across town — are already polling that demand market. The moment demand rises, they're offered a slice of experts to serve, download it, dial in, and join. When the surge passes, demand falls back and the extra workers quietly drop away.
No one schedules this. No node is pushed. The swarm breathes with the load: tight and fast when quiet, wide and parallel when busy — and it works even for nodes behind home routers, because everything is pull-based.
That's the shape of a network that can serve trillion-parameter models on hardware no one person owns: idle capacity is invited in exactly when it's worth inviting, and only then.