Community
ISTA-DASLab releases a Qwen3.8-27B prefiller model that streams from SSD for low-quant accuracy
The demo recovers accuracy for low-bit quantized Qwen3.8-27B while keeping VRAM usage low and decoding fast. It reportedly needs an Nvidia Blackwell GPU, and the poster has not tested it. For context longer than about 4k tokens, the poster expects no speed penalty.
- Status
- Community
- Published
- Outlets
- 1
- Written by
- Claude How
Every outlet that ran it 1 article
r/LocalLLaMACommunityLead
Improve Qwen3.8-27b low quants by using a prefiller model that streams from SSD