linkcpp

The source-available (BSL) control plane that turns everyday hardware into a distributed inference engine.

linkcpp is the engine behind Kvasir: a control plane around inference engine's RPC data plane that runs large AI models across multiple GPUs and machines using *stock* ggml-rpc-server / llama-server binaries. Everything it adds is orchestration — GPU discovery, node slots, layer-placement planning, worker launch, and the OpenAI/Anthropic gateways.

browser / SDK
  → hub :19000                      # FastAPI control plane (Docker)
  → GPU-less llama-server master    # per controller, :8080+
  → ggml-rpc-server workers         # slots :50052-50056 · units · agents

• Source-available under the BSL — free to read, run and build on in development and testing; production use requires a license.
• The inference engine data plane stays unforked (one pinned mobile GPU-over-RPC patch aside), so upstream performance work keeps flowing in.
• Ships as a single Docker image: the FastAPI hub plus the two inference engine binaries baked in; native worker nodes build outside Docker for CUDA/Metal/Vulkan/CPU.

The planner reads GGUF metadata and produces contiguous per-node layer windows, the matching --tensor-split, and KV-cache / layer / expert VRAM estimates per node — plus optional MoE expert-FFN offload to node RAM, emitted as inference engine -ot rules (e.g. blk\.38\.ffn_(up|down|gate)_(ch|)exps=CPU) and carried to launch via --override-tensor. A plan that doesn't fit is reported infeasible before anything loads, not discovered as an OOM at runtime.

Runtime compatibility is a first-class concept: protocol, runtime-pack, inference engine revision and RPC ABI are verified and mismatches hard-blocked before any bind, plan, load or inference.