ninfer-4090
Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention prefill, vision, llama.cpp-compatible /metrics + /slots
Take a look →OPEN SOURCE, IN FOCUS
rotorquant
scrya-com/rotorquant
KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44x fewer params. Drop-in llama.cpp integration.
Saved projects stay in this browser. No account needed.
TAKE A CLOSER LOOK
Start with the source.
The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.
Read the documentation on GitHub ↗Keep exploring
Find projects in the same areas, or search the archive for a different approach.