auto-round
A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.
Take a look →OPEN SOURCE, IN FOCUS
sparser-faster-llms
sakanaai/sparser-faster-llms
Cuda kernels for leveraging LLM sparsity to improve throughput and decrease the memory requirements during inference and training.
Saved projects stay in this browser. No account needed.
TAKE A CLOSER LOOK
Start with the source.
The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.
Read the documentation on GitHub ↗Keep exploring
Find projects in the same areas, or search the archive for a different approach.