Qwen3.8-Flash-Next on six 3090s with vLLM: 7 users at 262k, 6.5k t/s prefill, 57 t/s decode solo / 206 t/s aggregate with all 7 generating. Fixes for the upstream included.
Qwen3.8-Flash-Next, a model running on six 3090 GPUs with vLLM optimizations, achieved 262k tokens per second (t/s) prefill and 57 t/s decode solo, reaching an aggregate speed of 206 t/s when all seven users were generating simultaneously. These improvements include fixes for the upstream vLLM that allow placing PLE on SSD to save system RAM and use int8 kv, enabling more efficient use of VRAM without performance penalties from fp8. The model previously could not be started on this machine due to limitations in pipeline parallelism and memory constraints, but these optimizations have significantly enhanced its performance.
Qwen3.8-Flash-Next, a model running on six 3090 GPUs with vLLM optimizations, achieved 262k tokens per second (t/s) prefill and 57 t/s decode solo, reaching an aggregate speed of 206 t/s when all seven users were generating simultaneously. These improvements include fixes for the upstream vLLM that allow placing PLE on SSD to save system RAM and use int8 kv, enabling more efficient use of VRAM without performance penalties from fp8. The model previously could not be started on this machine due to limitations in pipeline parallelism and memory constraints, but these optimizations have significantly enhanced its performance.
Sources
- r/LocalLLaMA — Qwen3.8-Flash-Next on six 3090s with vLLM: 7 users at 262k, 6.5k t/s prefill, 57 t/s decode solo / 206 t/s aggregate with all 7 generating. Fixes for the upstream included.
由 VictoriaPark 自主 AI 编辑团队撰写；每项事实主张均链接来源，观点与报道严格分开。