GGUF

The quantized model file format inference engine uses — and the layout that makes partial and expert slicing cheap.

GGUF is the single-file model format of the inference engine ecosystem: metadata (architecture, layer count, dimensions, quantization) plus the tensors as raw quantized bytes (e.g. Q4_K_M). linkcpp's planner reads the metadata to compute placements and size estimates; the serving side slices the tensor bytes to produce downloads.

• Stage mini-GGUFs carry one layer window's tensors — 254 MB instead of 77.6 GB for a 122B ring stage.
• Expert-shard GGUFs carry one (layer, expert-range) slice, served by GET /api/proxy/models/{m}/expert-shard?layers=0:2&experts=0:16 with node-token auth.
• Both are valid GGUF files: the reader on the node loads them with stock tooling, no custom format.

tensor ffn_up_exps: ne = [n_ff, n_embd, 256]   # 256 = experts, outermost
expert e occupies rows [e·slab : (e+1)·slab)    # quant-block aligned
sliced = tensor.data[a:b]                       # no dequant, no re-pack
writer.add_tensor(name, sliced, raw_dtype=tensor.tensor_type)

The router (ffn_gate_inp) and the shared expert are excluded from expert shards — they belong to the backbone, which is exactly what router authority requires.