Fewer attention heads reduce verbatim memorisation in large language models
Read the original at arxiv.org→arXiv:2610.40974v1 Announce Type: new Abstract: Large language models have been shown to memorise and reproduce verbatim sequences from their training data, raising concerns about privacy leakage, intellectual...
Original headline: "Fewer Heads, Fewer Memories: Pruning Redundant Attention Heads Suppresses Verbatim Memorisation"
Coverage timeline
- Nov 2, 05:00 UTC arXiv cs.CL lead source Fewer Heads, Fewer Memories: Pruning Redundant Attention Heads Suppresses Verbatim Memorisation