HN · GitHub Signals

OPEN SOURCE, IN FOCUS

nemotron-30b-cpu-256k

synchopate/nemotron-30b-cpu-256k

Running a 30B MoE LLM on CPU at 30-50+ tok/s with a 256K Context Window via Hybrid Mamba-2 SSM, L3 NTA Slicing, AVX-512 VNNI Tile Repacking & 2MB HugePages

Explore on GitHub ↗

Saved projects stay in this browser. No account needed.

TAKE A CLOSER LOOK

Start with the source.

The repository is the place to check installation steps, supported platforms, the license, and current maintenance before choosing a tool.

Read the documentation on GitHub ↗

Keep exploring

Find projects in the same areas, or search the archive for a different approach.

ANOTHER GOOD FIND

Explore related projects

View the archive ↗

Connected by topics and descriptions.

hiyucat

deepseek-v4-pro-flash-desktop-app

deepseek-v4-pro-flash deepseek ai large language model llm mixture of experts moe 1m context window hybrid attention architecture compressed sparse attention csa heavily compressed attention hca manifold constrained hyper deepseek-v4-pro deepseek-v4-flash open source hugging face github repository api access local llm inference vllm ollama

Take a look →
guqiong96

lvllm

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

Take a look →
syv-ai

hyperqwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

Take a look →