Deep Learning, Artificial Intelligence, Medical, NLP, Llm.
F1-scores and Pearson scores. 2. Benchmarking Models & Results 2.1. BERT Pretraining Corpora In this paper, BERT is pre-trained on PubMed abstracts and clinical notes (MIMIC-III). The statistics of the text corpora on which BERT was pre-trained are shown in Table 2. BERT is initialized with