Why higher micro-batch values may improve llama-cpp model output quality, per user observations
Read the original at old.reddit.com→Or is there any reason why I feel like model output quality seems to be better when I use higher micro-batch values (ub) in llama-cpp? I don't really have any hard numbers or anything (just running the same prompts),...
Original headline: "Am I just hallucinating"
Coverage timeline
- Aug 7, 16:20 UTC r/LocalLLaMA lead source Am I just hallucinating