HN · GitHub Signals

OPEN SOURCE, IN FOCUS

ninfer-4090

sergiuszm/ninfer-4090

Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention prefill, vision, llama.cpp-compatible /metrics + /slots

Explore on GitHub ↗

Saved projects stay in this browser. No account needed.

TAKE A CLOSER LOOK

Start with the source.

The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.

Read the documentation on GitHub ↗

Keep exploring

Find projects in the same areas, or search the archive for a different approach.

ANOTHER GOOD FIND

Explore related projects

View the archive ↗

Connected by topics and descriptions.

dominikbucko

qwen38-flash-next-2x3090

Qwen3.8-Flash-Next on 2× RTX 3090 + 128 GB RAM. Experimental peaks: 1,860 tok/s prefill at 260K input; 89.1 tok/s decode at 128K input. Full 256K context.

Take a look →
don-chad

ninfer-3090

Fast Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows builds.

Take a look →
0xbakeer

qwen3.8-27b-4-bit-on-a-single-dgx-spark

Serving 4-bit Qwen3.8-27B on a single DGX Spark (GB10): 75 tok/s single-stream, 246 tok/s aggregate at 8-way concurrency. NVFP4 vs MixedInt4-AutoRound vs the FP8 baseline, measured on one harness — including why the quantization advantage collapses to +0.2% by c16.

Take a look →
jakeatx

llamampere

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): TurboQuant KV cache, MTP speculative decoding with a 64K draft-vocabulary shortlist, custom SM86 + Qwen kernels. 90 tok/s over a 100K-token generation at temperature 1.

Take a look →