Towards scalable lifelong knowledge editing with selective knowledge suppression

In this paper, the authors tackle continued pretraining without the risk of catastrophic forgetting, by identifying parameters which can safely be changed without risking identified concepts, and freezing the rest:...