kindi
A custom LLM inference engine for low-cost NVIDIA GPUs: Quadro K420 and K1200, two K1200s joined into one 8 GB card, GTX 1080 and RTX 3080. Runs Bonsai 2B, 4B, 9B and 27B models with an OpenAI-compatible server.
Take a look →OPEN SOURCE, IN FOCUS
mmaemu
astding/mmaemu
Bit-exact emulation of NVIDIA tensor-core FP8 GEMMs (torch._scaled_mm, Transformer Engine) as an H100 PCIe or RTX 4060 computes them, on the CPU or another CUDA GPU
Saved projects stay in this browser. No account needed.
TAKE A CLOSER LOOK
Start with the source.
The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.
Read the documentation on GitHub ↗Keep exploring
Find projects in the same areas, or search the archive for a different approach.