Overfitting limits creativity in large-language models, despite strong test performance

I remember great creativity from GPT-2. It was trained on a dataset that is orders smaller than current training datasets. The problem with overfitting a model to the training data is that it becomes unable to...