分析
这篇文章简要概述了自然语言处理（NLP）的进展，重点关注大规模预训练语言模型。它强调了GPT和BERT等模型的影响，并与计算机视觉中的预训练进行了类比。文章强调了预训练不需要标记数据的优势，从而可以进行更大规模的训练实验。更新显示了该领域进展的时间线，展示了不同模型的演变。
引用 / 来源
查看原文"Large-scale pre-trained language modes like OpenAI GPT and BERT have achieved great performance on a variety of language tasks using generic model architectures. The idea is similar to how ImageNet classification pre-training helps many vision tasks (*). Even better than vision classification pre-training, this simple and powerful approach in NLP does not require labeled data for pre-training, allowing us to experiment with increased training scale, up to our very limit."