p4

The engine behind Kvasir: an event-addressed protocol where agents own nodes, stage servers hold layers, and placement is something an operator states rather than something the network guesses.

p4 runs one model across several machines by cutting it into stages — contiguous slices of its layers — and giving each stage its own process. An agent owns the nodes on a host: it spawns stage servers, routes events between them, and answers for their lifecycle. There is no scheduler deciding where things go; an operator writes a placement plan, loads it, and the network then serves exactly that.

browser / SDK
  → gateway :8791              # payment, settlement, the wallet app
  → bridge :19000              # OUTER: session, submit, gather
  → p4 agent                   # owns this host's nodes
  → stage servers              # one process per layer slice

When a stage finishes its layers it hands the result to the next stage by asking its own agent to open a connection to that stage's agent, at the address that agent advertises. The advertised address is therefore not cosmetic: it has to be reachable from every other host in the ring, and it should name the fastest network they share. Advertise loopback and a two-host ring quietly dials itself.

• The load generation is chosen by whoever loads and is compared for exact equality on every session, inference, settlement and unload. It is recorded nowhere on the machines, so the loader writes it to disk *before* the first command leaves — without it a loaded model cannot even be taken down.
• A node's generation and the load generation are the same number. The adapter checks a release receipt's source generation against the load it belongs to and stops the node when they differ, so a ring loaded with two different numbers serves one request and then loses its head.
• An operational journal is required before a model will load at all: it is the admission record that makes a load replay-safe, not a debugging aid.

p4 applies no chat template — it forwards an opaque prompt and expects the caller to have rendered the model's turn format. It makes no placement decisions. And it carries no notion of who should be paid: stages report the token rows they ran, and settlement is somebody else's contract. Each of these is a seam Kvasir fills in the [bridge](/wiki/bridge), which keeps the engine narrow enough to track upstream.

• A ring gets slower the longer it runs. Every journal record asks whether it fits, and the answer is computed by listing the journal directory and stat-ing every entry; nothing is ever deleted from it, so each write pays for every record the agent has written since it started. Measured on two stages: 22.90 tok/s falling to 12.58 over four consecutive runs. Agent-to-agent hops write records a single-agent ring never writes, so a ring split across agents decays several times faster.
• The stage servers spin while the GPU works. All the arithmetic runs on the accelerator, but each stage starts a CPU thread pool the size of the machine's cores and the OpenMP runtime spins those threads while idle — two stages peg every logical core, and the agent's per-event work, which paces the ring, runs on what is left. OMP_WAIT_POLICY=PASSIVE took agent CPU per token from 116 ms to 8 ms.
• One agent per host, where the shape allows it. A hop inside an agent is a function call; a hop between two agents is a journalled, durable event. Four stages under one agent served 7.37 tok/s; the same four split across two agents on the same machine served 2.80.