LIVE · refreshes every 20 min

updated Oct 4, 00:43 UTC

← 110010.art all stories

r/LocalLLaMA ·5h ago

I built ninfer 4080 for 16gb class GPUs

Read the original at www.reddit.com→

Hi everyone, TL/DR I created NInfer 4080 to run ISTA-DASLab-Qwen-3.8-27B-GSQ at 100k context on an RTX 4080 16GB GPU using way more of the hardware capabilities (max overall: 2720 tok/s prefill, 262 tok/s generation)...

Original headline: "I built Ninfer 4080 for 16GB class GPUs"

Coverage timeline

Oct 3, 18:58 UTC r/LocalLLaMA lead source I built Ninfer 4080 for 16GB class GPUs

Related stories

CUDA: fuse shared experts into MMVQ in ggml-org/llama.cpp; MoE speedup shown for select architectures like Qwen 35B A3B

r/LocalLLaMA · 1d ago · source ↗

Jeff-Qwen3.5-0.8B v1.2 + 9 LoRA adapters front-ends Qwen3.8-27B for 38× faster decisions and +8.7 points accuracy with under 2 GB extra memory

r/LocalLLaMA · 2d ago · source ↗

Local models as alternatives to Qwen/Deepseek for coding and long-document QA; evaluating Gemma 31B, Muse Glimmer 30B, Nemotron, and fine-tuning impacts vs Qwen3.8 27B

r/LocalLLaMA · 1d ago · source ↗

Two open-weight releases: Victoria Qwen3.8-Flash-Next with 44% expert cuts and 70% Terminal-Bench 2.1; Maple Canada-first fine-tune

r/LocalLLaMA · 3d ago · source ↗

Alibaba releases Qwen 3.8 Omni Flash

Hacker News (AI) · 16d ago · source ↗