LIVE · refreshes every 20 min

updated Oct 19, 18:23 UTC

110010.art / topic

CUDA

Oct 19

5h ago Intel D270 GPU: specs include up to 480 Gb VRAM LPDDR5, 1.5T/s memory bandwidth, and interference optimized; user asks about pricing, workstation vs server use, and driver support r/LocalLLaMA → story

Oct 17

2d ago CUDA shows big speedup for low-bit MMVQ types like Q2_K and Q3_K in llama.cpp pull request 30517 r/LocalLLaMA → story

Oct 16

3d ago Drama3 in audio.cpp runs up to 5× faster than real time at full speed; 2× real-time on RTX 5090 with <8 GB VRAM, CUDA-focused optimizations noted r/LocalLLaMA → story

Oct 14

5d ago Moving a running model’s layers between GPUs mid-reply (8 ms, no re-prefill) r/LocalLLaMA → story

Oct 13

6d ago SonicBoom achieves 52 tok/s decode on a laptop RTX 3050 (llama.cpp 45) with the same quantized kernels; the difference is a graph compiler, with no Python involved. r/LocalLLaMA → story

6d ago Modders discuss using AI to reverse-engineer CUDA implementations for open-source GPU tooling r/LocalLLaMA → story

Oct 12

6d ago Lecture: Building a GPU compiler from scratch; lowering PyTorch graphs to CUDA via multi-level IRs r/LocalLLaMA → story

7d ago Show HN: Reflex is a GGUF-native Rust/CUDA engine for scale-to-zero LLM routing Hacker News (AI) → story

Oct 11

7d ago GitHub project Decapybara provides open-source C++20 and CUDA reconstruction of the MegaCapybara inference server r/LocalLLaMA → story

8d ago Now you can grow Bonsai on your potato r/LocalLLaMA → story

8d ago Local semantic file search for Linux uses EmbeddingGemma 2 with Rust, llama.cpp, and in-process embedding across text, images, video frames, and audio windows; hybrid ranking combines vector similarity with SQLite FTS5 r/LocalLLaMA → story

Oct 8

10d ago Blackwell GPUs experience PCIe 5 issues, according to user reports and diagnostic details r/LocalLLaMA → story

11d ago Java vllm-like framework achieves ~90% of llama.cpp performance for local inference on NVIDIA GPUs by compiling Java to CUDA and cuTile r/LocalLLaMA → story

Oct 4

14d ago Vulkan-friendly GPUs for budget builds; list of not-supported-anymore GPUs with recommendations and tested options (GTX 1080 Ti, P102-100, MI50) r/LocalLLaMA → story

15d ago LLM inference dashboard offers local, scalable logging and metrics across multiple engines; not public yet but seeks interest r/LocalLLaMA → story

Oct 2

16d ago CUDA: fuse shared experts into MMVQ in ggml-org/llama.cpp; MoE speedup shown for select architectures like Qwen 35B A3B r/LocalLLaMA → story

Oct 1

17d ago DDR4/PCIe4 vs DDR5/PCIe5 for LLM pre-training: a benchmarking comparison for AI workstations r/LocalLLaMA → story

Sep 27

22d ago Mem v3 acts as a memory governor for PyTorch, dynamically adjusting batch size and gradient accumulation to prevent CUDA OOMs during training and fine-tuning r/LocalLLaMA → story

Sep 25

24d ago Show HN: Agentic CUDA kernel optimizer Hacker News (AI) → story

Sep 24

25d ago XLOG: a CUDA-native engine for neurosymbolic integration arXiv cs.AI → story

Sep 22

27d ago NVIDIA Isaac ROS uses an AI agent to accelerate a ROS 2 node; GPU speedups do not guarantee a faster ROS 2 graph. NVIDIA Developer → story