Berth, a lightweight Go-based GPU resource manager, coordinates GPU access via leases, priorities, and a queue to manage when apps use VRAM and hand control to the next task.
Read the original at www.reddit.com→Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off. Instead of letting every application compete for...
Original headline: "Live-Swap CUDA Workloads: llama.cpp, ComfyUI, Strata & More"
Coverage timeline
- Nov 2, 03:36 UTC r/LocalLLaMA lead source Live-Swap CUDA Workloads: llama.cpp, ComfyUI, Strata & More