HN · GitHub Signals

OPEN SOURCE, IN FOCUS

llama.cpp-tq3

turbo-tan/llama.cpp-tq3

llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired Walsh-Hadamard transform. Enables 27B models on 16GB GPUs with 15 tok/s TG, 221 tok/s PP.

Explore on GitHub ↗

Saved projects stay in this browser. No account needed.

TAKE A CLOSER LOOK

Start with the source.

The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.

Read the documentation on GitHub ↗

Keep exploring

Find projects in the same areas, or search the archive for a different approach.

ANOTHER GOOD FIND

Explore related projects

View the archive ↗

Connected by topics and descriptions.

aiksa2090

agentic-swarm

🧠 Multi-Agent AI System running 100% offline using Google Gemma-4 models. 5 specialized AI agents collaborate in infinite loops for brainstorming. Features: Dynamic VRAM swapping, 4-bit NF4 quantization, WhatsApp-style RTL UI, live token streaming, AI-powered agent customization. Built with PySide6 + PyTorch + Transformers.

Take a look →
futuremls-lab

oscar

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization

Take a look →
kacper-daftcode

vllm-moet

A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official (NV)FP4 checkpoint's quality on consumer Blackwell cards

Take a look →
jakeatx

llamampere

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): TurboQuant KV cache, MTP speculative decoding with a 64K draft-vocabulary shortlist, custom SM86 + Qwen kernels. 90 tok/s over a 100K-token generation at temperature 1.

Take a look →