Matching stories Newest first
Eagle-4 speculative decoding method reported faster than DFlash2 and DSpark on Qwen3.8-27B
80x. cpp and vLLM support later.
Reddit user runs Qwen 3.8 Flash Next locally on an 8GB GPU under CachyOS
The poster reports about 30 tokens per second on average and bursts up to 48 tokens per second, using a 3-bit Unsloth quant with a 42k context that nearly exhausts VRAM.
Simon Willison describes Battle Drones, a vibe coded RTS built with Claude
Willison built the browser RTS mostly with coding agents, mainly Claude Code, over 12 iterations and 321 commits. It includes audio-based multiplayer for two devices in microphone range, and he says he reviewed almost none of the code.
DeepSeek open-sources DeepInfer, the serving stack it runs in production
It ships under Apache 2.0 and for now supports only DeepSeek-V4.1-Flash in offline mode. It leads Hacker News with 1,862 points.
Berth open-source tool lets a single RTX 4090 be shared between local AI engines
Berth is a system service that swaps and loads local AI engines such as llama-swap, strata and ComfyUI on one GPU, and can serve them as an API. The project is shared as a GitHub repo.
HermesConsole and VoiceManager open-source a low latency local speech-to-speech demo
A hobbyist released VoiceManager, a streaming STT and TTS server under MIT, with a Python client and a Kotlin console reference app. Barge-in is off by default because echo cancellation is hard, and the default setup uses Nemotron and Breeze TTS 2.