A local AI engine and server for one NVIDIA DGX Spark. It serves Qwen3.8-27B and Qwen3.8-Flash-Next with its own NVFP4 and EXL3 kernels, draft trees sized by StairCut, and caches that survive a restart. OpenAI-compatible API, chat, dashboard and a setup guide.
LLM server for NVIDIA Blackwell only (GeForce RTX 50, RTX PRO). On one RTX 5090, Qwen3.8-27B one user: 96.3 tok/s, 1.38x vLLM without speculative decoding, 1.23x llama.cpp on the same GGUF. Serves the 123.6 GiB Qwen3.8-Flash-Next from a 32 GB card at 92.1 tok/s. OpenAI and Anthropic API, tool calls, one docker run.
Run Kimi K3, a 2.8T-parameter Mixture-of-Experts LLM, on a single Apple Silicon Mac. Streams MXFP4 experts on demand over HTTP into a local disk cache — fused NEON kernels, Metal/MPS compute, exact reproducible decoding, and an OpenAI-compatible API server for local chat and coding agents.
GLM-5.3-Flash (abliterated EXL3) on 2x NVIDIA DGX Spark with the TensorFold engine: 1.8x faster decode than vLLM, 4x256k concurrent threads, byte-exact speculative decoding. Work in progress.