Community · Oct 26, 2026, 07:49 UTC

Ternary Nemotron 3.5 Lightning 30B-A3B GGUF released on Hugging Face

A 1.58-bit ternary 30B-A3B MoE model in GGUF format that the poster says runs on a 12 GB GPU with 368K context, 172 tok/s decode and over 7000 tok/s prefill, on stock llama.cpp and the PrismML fork. Performance figures are the poster's own claims.

• LabNVIDIA
• ModelTernary-Nemotron-3.5-Lightning-30B-A3B
• Topicrelease
• Topicopen-weights