  BERT (Bidirectional Encoder Representations from Transformers) and ALBERT (A
Lite BERT) are methods for pre-training language models which can later be
fine-tuned for a variety of Natural Language Understanding tasks. These methods
have been applied to a number of such tasks (mostly in English), achieving
results that outperform the state-of-the-art. In this paper, our contribution
is twofold. First, we make available our trained BERT and Albert model for
Portuguese. Second, we compare our monolingual and the standard multilingual
models using experiments in semantic textual similarity, recognizing textual
entailment, textual category classification, sentiment analysis, offensive
comment detection, and fake news detection, to assess the effectiveness of the
generated language representations. The results suggest that both monolingual
and multilingual models are able to achieve state-of-the-art and the advantage
of training a single language model, if any, is small.

Computation and Language;Machine Learning