Agens Volundr 32B Preview: our small team's first model on our own hybrid architecture; only 18 of 72 layers keep a KV cache
Read the original at www.reddit.com→Hi r/LocalLLaMA. I'm on the team at Blockway, a small team in Hong Kong (disclosure: this is our model). Today we released Agens Volundr 32B Preview, the first model built on our own hybrid architecture. We trained...
Original headline: "Agens Volundr 32B Preview: our small team's first model on our own hybrid architecture. Only 18 of 72 layers keep a KV cache (Apache-2.0)"
Coverage timeline
- Oct 5, 12:58 UTC r/LocalLLaMA lead source Agens Volundr 32B Preview: our small team's first model on our own hybrid architecture. Only 18 of 72 layers keep a KV cache (Apache-2.0)