readme: ---
  language: ro
  inference: true
  license: apache-2.0
  
  tags:
  - romanian
  - seq2seq
  - t5
  ---
  
  This is the fine-tuned [mt5-base-romanian](https://huggingface.co/dumitrescustefan/mt5-base-romanian) base model (**390M** parameters).
  
  The model was fine-tuned on the [romanian diacritics dataset](https://huggingface.co/datasets/dumitrescustefan/diacritic) for 150k steps with a batch of size 8. The encoder sequence length is 256 and the decoder sequence length is also 256. It was trained with the following [scripts](https://github.com/iliemihai/t5x_diacritics).
  
  ### How to load the fine-tuned mt5x model
  
  ```python
  from transformers import MT5ForConditionalGeneration, T5Tokenizer
  model = MT5ForConditionalGeneration.from_pretrained('iliemihai/mt5-base-romanian-diacritics')
  tokenizer = T5Tokenizer.from_pretrained('iliemihai/mt5-base-romanian-diacritics')
  input_text = "A inceput sa ii taie un fir de par, iar fata sta in fata, tine camasa de in in mana si canta nota SI."
  inputs = tokenizer(input_text, max_length=256, truncation=True, return_tensors="pt")
  outputs = model.generate(input_ids=inputs["input_ids"], attention_mask=inputs["attention_mask"])
  output = tokenizer.decode(outputs[0], skip_special_tokens=True)
  print(output)  # this will print "A început să îi taie un fir de păr, iar fata stă în față, ține cămașa de in în mână și cântă nota SI" 
  ```
  
  ### Evaluation
  
  Evaluation will be done soon [here]()
  
  ### Acknowledgements
  
  We'd like to thank [TPU Research Cloud](https://sites.research.google/trc/about/) for providing the TPUv3 cores we used to train these models!
  
  ### Authors
  
  Yours truly,  
  
  _[Stefan Dumitrescu](https://github.com/dumitrescustefan), [Mihai Ilie](https://github.com/iliemihai) and [Per Egil Kummervold](https://huggingface.co/north)_
  
  