readme: ---
  language: pt
  license: mit
  tags:
    - bert
    - pytorch
  datasets:
    - Twitter
  ---
  
  
  
  ## Introduction
  
  Having the same architecture of [Bert] we trained our model from scratch following [BERT](https://arxiv.org/abs/1810.04805) pre-training procedure. And has been built from a collection of about 238 million tweets written by over 100 thousand unique Twitter users, and conveying over 2.9 billion words in total.
  
  ## Available models
  
  | Model                                    | Arch.      | #Layers | #Params |
  | ---------------------------------------- | ---------- | ------- | ------- |
  | `pablocosta/bertabaporu-base-uncased`    | BERT-Base  | 12      | 110M    |
  | `pablocosta/bertabaporu-large-uncased`   | BERT-Large | 24      | 335M    |
  
  ## Usage
  
  ```python
  from transformers import AutoTokenizer  # Or BertTokenizer
  from transformers import AutoModelForPreTraining  # Or BertForPreTraining for loading pretraining heads
  from transformers import AutoModel  # or BertModel, for BERT without pretraining heads
  model = AutoModelForPreTraining.from_pretrained('pablocosta/bertabaporu-large-uncased')
  tokenizer = AutoTokenizer.from_pretrained('pablocosta/bertabaporu-large-uncased')
  ```