Phase 4 — The Expert-Slice Data Path (M1)

A weak device downloads a few 6 MB experts, not a 1.4 GB layer — and sharded compute matches monolithic to 3.6e-12.

MoE expert tensors stack all experts along the outermost ggml dimension, so the reader exposes (n_expert, rows, row_bytes) of raw quantized bytes. Expert *e* is a quant-block-aligned contiguous slab — the slice is literally data[a:b], with no dequantization and no re-packing.

sliced = tensor.data[a:b]              # outermost axis = expert
writer.add_tensor(name, sliced, raw_dtype=tensor.tensor_type)
# router (ffn_gate_inp) & shared expert stay on the backbone → excluded
GET /api/proxy/models/{m}/expert-shard?layers=0:2&experts=0:16  # node-token authed

With real 122B layer-0 experts (dequantized reference), splitting the experts into 4 shards, computing each separately and combining matches the monolithic MoE FFN: sharding is an exact regrouping of the same weighted sum, not an approximation.

• Pure ggml/gguf (no libllama): loads the slice into a GPU backend and runs mul_mat_id(up/gate) → swiglu → mul_mat_id(down).
• ROCm build + run on an MI250: 122B layer-0, experts [0,8), 4 tokens.
• Cosine 0.99995 vs the oracle, allclose(1e-3) = True, max|Δ| = 7.9e-7 — this residual is itself the first measured instance of cross-backend equivalence (ROCm vs numpy).
• The same code path covers CUDA/Metal/Vulkan/CPU (mul_mat_id/swiglu are stock ggml; CUDA has a dedicated MoE kernel).

The hardest, riskiest piece — the on-device worker kernel — was verified here. What remained was backbone↔worker orchestration; the worker is a proven pure function consuming these slices.