The first pure SNN language model trained from scratch with a fully original architecture. 618M parameters • 93% sparsity • Runs on phone • Online learning via STDP • $260 total training cost
🕹️ Is this the smallest language model in the world? I just managed to squeeze real artificial intelligence into 30 kilobytes, running on a 1979 Atari 800. A fully generative, deterministic language model, powered by a neural network and built to run on 8‑bit hardware.