I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens
Read the original at www.reddit.com→DISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and hashed n-gram...
Coverage timeline
- Oct 11, 10:19 UTC r/LocalLLaMA lead source I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens