ExL3-QLoRA trains 2-8 bpw exLlamaV3 quantized models using plain PyTorch; adapters saved in standard PEFT format
Read the original at www.reddit.com→EXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without transformers....
Original headline: "QLoRA on ExLlamaV3 Quants - Train @ 2-8bpw"
Coverage timeline
- Oct 12, 07:30 UTC r/LocalLLaMA lead source QLoRA on ExLlamaV3 Quants - Train @ 2-8bpw