Ring runtime

Pipeline inference without a master: each device runs its layer window and passes only boundaries to its neighbor.

The ring runtime is Kvasir's low-latency serving topology. Every device loads only its contiguous layer window, then opens exactly two links — predecessor and successor. Hidden-state boundaries circulate around the ring; the last rank samples the token and returns it. No central master, and no node holds the whole model.

In the classic RPC topology one master opens the entire GGUF and dials out to every worker. That breaks in an open network three ways: the master must hold and serve the whole checkpoint; every worker must be dialable — phones behind carrier NAT are not; and the master is a single owner in a network that should have none. The ring removes all three: each stage owns its window, connections are neighbor-to-neighbor, and the relay makes NAT devices reachable.

token n:  stage A (layers 0-14)  ──h──▶  stage B (15-26)
                                             │h
          stage D (37-48) ◀──h──  stage C (27-36)
          └─ samples token n, sends it around → client

• Placement comes from the planner's rank manifest — e.g. Qwen3.5-122B's 49 layers split across a GPU, CPU, NPU and phone.
• Boundaries are small (a hidden-state vector per token), so hops are cheap even for weak links.
• Mobile GPUs run ring stages directly (Adreno via OpenCL) — the RPC path to a phone GPU was infeasible because Adreno's buffer layout doesn't survive RPC serialization, but a local stage owns its backend, so only boundaries cross the wire.

The ring is the latency path; its floor is layer granularity (~1.4 GB on the 122B). The expert-sharded swarm removes that floor and plugs into the same serving fabric.