Community
RustingStrata Rust inference engine runs Qwen models on 12 GB GPUs
A pure Rust engine with NVRTC-compiled CUDA and Vulkan kernels runs Qwen GGUF models, reporting about 36 tok/s decode on Qwen3.8-Flash-Next IQ2_XS versus 19-22 for the original Strata, and 118 tok/s decode on Qwen3.6-35B-A3B where llama.cpp gets 68. These are the author's own benchmarks on a 3060 1…