LIVE · refreshes every 20 min

updated Oct 19, 16:24 UTC

110010.art / topic

Llama

Oct 3

16d ago MiniPC to run a LLM w/ voice assistant on Ubuntu 24.04; llama.cpp and Python setup for offline, local voice-first assistant on a fanless mini PC r/LocalLLaMA → story

Oct 2

16d ago CUDA: fuse shared experts into MMVQ in ggml-org/llama.cpp; MoE speedup shown for select architectures like Qwen 35B A3B r/LocalLLaMA → story

16d ago −2 logit bias on Bonsai 2 27B reduced accuracy from 44/50 to 43/50 on MATH-500; tokens slightly increased in run B r/LocalLLaMA → story

17d ago Llama.cpp adds decision models r/LocalLLaMA → story

17d ago llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) r/LocalLLaMA → story

17d ago Ninfer Stahp runs qwen3.8-27b-nvfp4 on Blackwell MaxQ, delivering 130–180 t/s and strong parallel decoding performance vs llama-server, with Windows 10 setup underway r/LocalLLaMA → story

17d ago Format-aware fusion for fast FP4 pretraining improves efficiency by co-designing quantization producers with their scale domains and consumer layouts for native MXFP, global NVFP, and cooperative-thread-array-local NVFP arXiv cs.LG → story

17d ago Update on the “Monstrosity”: 6 BC-250 board cluster testing Qwen Next Flash IQ2_XS and 3.6 35b Q4 setups with llama, Vulkan, and RPC over 1gb Ethernet r/LocalLLaMA → story

Oct 1

17d ago cobanov/PS5LM: llama.cpp on a jailbroken PS5 runs any GGUF model natively on the console, starting with Qwen 3.8 GitHub Trending (AI) → story

18d ago Unsloth, Swift1.5, Peculiar-Ragdoll, ThinkingCap - Qwen3.8-27B comparison across 69 eval questions at medium reasoning effort r/LocalLLaMA → story

18d ago Pi extension forces local Qwen 27B to skip reasoning and answer immediately r/LocalLLaMA → story

18d ago Qwen Flash Next MTP work restarted r/LocalLLaMA → story

18d ago A PCB with no chips costs $500 r/LocalLLaMA → story

Sep 30

18d ago Open source inference engine optimizes itself for your exact hardware, compiles and tunes kernels on-device, delivering up to 2x faster open models than llama.cpp on Apple Silicon, NVIDIA, AMD, or CPU-only setups r/LocalLLaMA → story

18d ago Strix Halo + R9700 with llama-halo-hybrid now outperforms DGX Spark r/LocalLLaMA → story

19d ago Llama.cpp adds GLM-5.3-Flash (GLM5-Next) support; users can run GLM-5.3-Flash on home computers. r/LocalLLaMA → story

19d ago Qwen3.8 achieves 50 t/s on Strata with 12GB VRAM and 64GB RAM laptop; Strata inference engine reports 1500 t/s PP on 64GB RAM featud? r/LocalLLaMA → story

19d ago Show HN: Reflex Engine beats both Llama.cpp and vLLM on cold-start to TTFT Hacker News (AI) → story

Sep 28

21d ago X account claims high t/s setup but provides few details r/LocalLLaMA → story

21d ago Would you buy an AI appliance that handles infrastructure for local models like Qwen 3.8 27B and Gemma 3 27B? r/LocalLLaMA → story

21d ago Llama.cpp MacOS menu bar app uses blobs instead of GGUF files for downloaded models. r/LocalLLaMA → story

Sep 27

21d ago Dual Radeon improves speeds with Vulkan r/LocalLLaMA → story

22d ago Advice on models for a RAG use case: selecting open-weight models for ingestion and querying on CPU-only setup r/LocalLLaMA → story

Sep 24

25d ago From decorative to load-bearing: task difficulty shapes the causal role of chain-of-thought arXiv cs.AI → story

Sep 22

27d ago Evaluating fine-tuned and base language models for maternal health and vaccination in Nigerian settings; HelpMum's MamaBot-Llama and Vax-Llama vs Meta's Llama-3.1-8B-Instruct arXiv cs.CL → story

27d ago Transformers now runs llama.cpp quantized models Hugging Face → story

← 1 2 3 →