HN · GitHub Signals

OPEN SOURCE, IN FOCUS

vllm.cpp

mudler/vllm.cpp

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features

Explore on GitHub ↗

Saved projects stay in this browser. No account needed.

TAKE A CLOSER LOOK

Start with the source.

The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.

Read the documentation on GitHub ↗

Keep exploring

Find projects in the same areas, or search the archive for a different approach.

ANOTHER GOOD FIND

Explore related projects

View the archive ↗

Connected by topics and descriptions.

aeon-7

vllm-ultimate-dgx-spark

AEON vLLM Ultimate — vLLM 0.24.0 built from source for DGX Spark / Blackwell (sm_121a/GB10). One image serves the whole AEON fleet (Gemma-4-26B-A4B, Qwen3.6-27B, Qwen3.6-35B-A3B) with DFlash speculative decoding, NVFP4 weights + Triton NVFP4/FP8 KV cache, high-concurrency DFlash fix, UMA cudagraph clamp, FlashInfer 0.6.12.

Take a look →
aiha-lab

tangram

A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM

Take a look →
cloudrift-ai

emmy

Benchmark and deploy optimized LLM models on GPU servers with vLLM or SGLang. Chose from a list of optimized recipes for popular models or create your own with custom configurations. Run benchmarks across different GPU types and configurations, track results, and share experiments with the community.

Take a look →
entrpi

ds4

CUDA Blackwell performance fork of antirez/ds4 (DeepSeek-V4-Flash): ~4x prefill, ~1.5x decode, continuous batching, weight server, DSpark

Take a look →