readme: ---
  language: id
  datasets:
  - oscar
  ---
  # IndoBERT (Indonesian BERT Model)
  
  ## Model description
  IndoBERT is a pre-trained language model based on BERT architecture for the Indonesian Language. 
  
  This model is base-uncased version which use bert-base config.
  
  ## Intended uses & limitations
  
  #### How to use
  
  ```python
  from transformers import AutoTokenizer, AutoModel
  tokenizer = AutoTokenizer.from_pretrained("sarahlintang/IndoBERT")
  model = AutoModel.from_pretrained("sarahlintang/IndoBERT")
  tokenizer.encode("hai aku mau makan.")
  [2, 8078, 1785, 2318, 1946, 18, 4]
  ```
  
  
  ## Training data
  
  This model was pre-trained on 16 GB of raw text ~2 B words from Oscar Corpus (https://oscar-corpus.com/). 
  
  This model is equal to bert-base model which has 32,000 vocabulary size. 
  
  ## Training procedure
  
  The training of the model has been performed using Google’s original Tensorflow code on eight core Google Cloud TPU v2.
  We used a Google Cloud Storage bucket, for persistent storage of training data and models.
  
  ## Eval results
  
  We evaluate this model on three Indonesian NLP downstream task:
  - some extractive summarization model
  - sentiment analysis
  - Part-of-Speech Tagger
  it was proven that this model outperforms multilingual BERT for all downstream tasks.
  