LIVE · refreshes every 20 min

updated Aug 18, 11:22 UTC

110010.art / topic

Llama

Aug 18

just now Combining MTP with ngram-mod to improve repeated token output in coding tasks; user tests show bursts of output when repeating a code block with spec-type and spec-ngram-mod settings r/LocalLLaMA → story

7h ago DumpsterCluster: using retired GPUs to serve LLaMA-70B on 60 GPUs arXiv cs.LG → story

Aug 17

16h ago Qwen 3.8 27B outperforms alternatives and raises pricing concerns; user questions its reasonableness and notes continued Alibaba training after prior release r/LocalLLaMA → story

17h ago Llama.cpp adds adaptive MTP mode using a counting-style state machine to dynamically determine MTP depth; aims to automate depth selection as content density varies r/LocalLLaMA → story

21h ago llama.cpp releases semantic version v0.1.0 r/LocalLLaMA → story

21h ago EXL3 fades from r/LocalLLaMa conversations as its VRAM-focused, OpenAI-compatible API deployment (TabbyAPI) limits practical value on GPUs under 24 GB. r/LocalLLaMA → story

1d ago Ling 3.0 support merged into llama.cpp r/LocalLLaMA → story

1d ago Qwen3.8-27B Q8_0 on Strix Halo demonstrates strong performance on a local test of a single-page HTML flight simulator generation r/LocalLLaMA → story

Aug 16

1d ago Llama.cpp upgrades ROCm to 7.14 for Radeon 780m iGPU benchmarks; ROCm 7.14 adds gfx1103 support, but prebuilt binaries lack the target, requiring a source build r/LocalLLaMA → story

1d ago Qwen3.8 27B Q2 vs Q3 and Qwen3.6 35B-A3B MoE on 12GB VRAM r/LocalLLaMA → story

1d ago Let’s all thank Georgi Gerganov who gave use llama.cpp r/LocalLLaMA → story

1d ago Qwen 3.8 27b—Acquarium panel failure; Q8_K_XL_unsloth r/LocalLLaMA → story

2d ago Qwen 3.8 27b shows 22 t/s performance with Q4, max context, llama.cpp, and MTP enabled; user reports and configurations posted on Reddit r/LocalLLaMA → story

2d ago Best setup for a 16 GB VRAM + 128 GB RAM system for running LLMs like Qwen 3.6 35B and related GGUF quantizations; user tests with 12700k, 5060 Ti, and 128 GB RAM explored performance. r/LocalLLaMA → story

2d ago Show-off Saturday: Intel Arc B140 build with 10-core Xeon W-2255, 64GB RAM, and llama.cpp SYCL back-end r/LocalLLaMA → story

2d ago Redstart: a local AI server using Llama.cpp seeks feedback on its two applications, Redstart-N and another component r/LocalLLaMA → story

2d ago Local privacy focus on cache AI CLI and llama.cpp branch + Qwen3.x Hacker News (AI) → story

Aug 15

2d ago Windows Manager for llama.cpp offers a visual interface to manage runtimes, switch models, and run multiple local endpoints without scripts or manual server management r/LocalLLaMA → story

2d ago Qwen 3.8 27B Q8 faster than Q6 with MTP on Apple Silicon using llama.cpp and lmstudio gguf r/LocalLLaMA → story

2d ago ggml-org/llama.cpp adds Kimi-K3 text model via pwilkin in Pull Request #26185 r/LocalLLaMA → story

3d ago Quantized AI9Stars' G9v3-39A5B to GGUF and created a llama.cpp fork to support it r/LocalLLaMA → story

Aug 14

3d ago RetroCraft uses Qwen 3.8 27B Q8_K_XL for a one-shot demo with dual 3090s showing exact performance data r/LocalLLaMA → story

3d ago Open labs embrace continued post-training on existing models to achieve performance gains without retraining new base models r/LocalLLaMA → story

3d ago Llama.cpp server can run tools in rootless sandboxed containers via --tools option r/LocalLLaMA → story

3d ago KV cache tricks for agents: prompt caching in llama.cpp stores memory-resident KV cache and ejects on prompt delta; used to run subagents when only one large model fits on GPU r/LocalLLaMA → story

Aug 13

4d ago DS4 cloud vs. Qwen3.6 36B vs. Muse Glimmer 30B speed benchmark on Llama.cpp (RTX 5080) r/LocalLLaMA → story

4d ago Fine-tuned Qwen2.5-Coder-1.5B on 125k command pairs can write shell commands on a laptop CPU in about 1 second; runs via llama.cpp on 1.6GB RAM with 31.9 tok/s and 0.59s median per query r/LocalLLaMA → story

4d ago Qwen3.8-2.4T-A95B runs locally on RTX 5090 + RTX 5060 Ti at about 0.80 tokens per second using llama.cpp with Unsloth GGUF quantization and 512 routed experts, 10 active per token. r/LocalLLaMA → story

5d ago ggml-cpu/ops vectorizes flash-attention F16 to F32 conversion using F16C intrinsics; PR #26947 r/LocalLLaMA → story

5d ago Open-source LLMs for a 5060 Ti 16GB GPU: evaluating model options and throughput on Qwen2.5-14B with 32768 context r/LocalLLaMA → story

Aug 12

5d ago Gemma 4 QAT handles KV cache quantization significantly better, according to KLD benchmarks comparing non-QAT and QAT configurations. r/LocalLLaMA → story

6d ago Tested Nemotron 3.5 Lightning locally on coding; Hermes Agent shows strong tool-calling capabilities but overall code quality below expectations for its size r/LocalLLaMA → story

6d ago Qwen3.5-0.8B ran on an Amlogic A113X sub-$20 CPU with 512MiB memory, measured on-device with a custom C runtime and single static binary. r/LocalLLaMA → story

6d ago Cracks in the foundation: four minor architectural decisions reduce long context extensibility across Olmo, Llama, and Qwen dense models arXiv cs.CL → story

6d ago Tinkerer shares personal upgrades for local agents, including switching to llama.cpp and exploring harnesses like hermes agent and pi.dev r/LocalLLaMA → story

Aug 11

6d ago Ling-3.0 support added to llama.cpp in a 40-line PR for Tiny model; PR 26608 still not merged to mainline r/LocalLLaMA → story

6d ago Are third-party gguf/mmproj safe on Llama in production environments? Hacker News (AI) → story

6d ago Built a low-power llama.cpp server with an Intel N100 and RTX 5060Ti r/LocalLLaMA → story

7d ago Add CI targets for ROCm 7.14 in llama.cpp; PR 25775 adds Linux and Windows targets for ROCm 7.14 support r/LocalLLaMA → story

7d ago Llama-CPP: parallel agents decode well, but one agent’s prefill stalls all others during web searches r/LocalLLaMA → story

← 1 2 3 4 →