New Paradigm of Natural Language Processing: A Method Based on Pre-Trained Models
CHE Wanxiang, LIU Ting
(Harbin Institute of Technology, Harbin 150001, China)
Abstract: Pre-trained language models based on super-large-scale raw corpora, represented by BERT and GPT, can make full use of big models, big data, and big computing, which has significantly improved the performance of almost all-natural language processing tasks. The performances have reached or exceeded the human level on some datasets. Pre-trained language models have become a new paradigm for natural language processing. It is believed that in the future, natural language processing and even the entire field of artificial intelligence will continue to move forward along the path of “homogenization” and “scale”, and will integrate more sources of “knowledge”, such as multi-modal data, embodiment data, and social interaction data. Consequently, these methods will pave the way for achieving true general artificial intelligence.
Keywords: artificial intelligence; natural language processing; pre-trained language model; homogenization