Community
Community guide runs 262K context Qwen-Flash-Next Coder on an 8 GB GPU at 70 tokens per second
The setup uses a 3.5 bits-per-weight ISTA DASLab quant with selective MoE offloading to RAM and n-gram data on SSD, reaching about 70 T/s prompt processing and 8 T/s generation over PCIe 3.0.
Every outlet that ran it 1 article
r/LocalLLaMACommunityLead
8 GB GPU, 262K context 3.5 BPW Qwen-Flash-Next Coder 70 T/s using llama.cpp