representations. BERT consists of two stages: pre-training
and ﬁne-tuning. During pre-training, the model was trained
on unlabeled data using a variety of pre-training tasks. For
ﬁne-tuning, the BERT model is ﬁrst initialized with the
pre-trained parameters, and then all of the parameters are
ﬁne-tuned using labeled data from the downstream jobs. The
architecture of the BERT model is shown in ﬁgure 10.
A_Comprehensive_Review_on_Fake_News_Detection_With_Deep_Learning