Open_rknpu2_lm_infer runs an open NPU inference engine on Rockchip RK3588, enabling Qwen3.5-4B at ~10 tok/s without proprietary libraries.
Read the original at www.reddit.com→If you have an RK3588 board, here’s another way to put its NPU to work. I made open_rknpu2_lm_infer, a local LLM inference engine for the Rockchip RK3588 NPU. It runs without Rockchip’s proprietary userspace...
Original headline: "I built an RK3588 NPU runtime without any proprietary inference libraries: Qwen3.5-4B at ~10 tok/s, ~2x faster than vendor SDK."
Coverage timeline
- Oct 20, 21:03 UTC r/LocalLLaMA lead source I built an RK3588 NPU runtime without any proprietary inference libraries: Qwen3.5-4B at ~10 tok/s, ~2x faster than vendor SDK.