readme: ---
  language: 
  - bn
  licenses:
  - cc-by-nc-sa-4.0
  ---
  
  # banglat5_banglaparaphrase
  
  This repository contains the pretrained checkpoint of the model **BanglaT5** finetuned on [BanglaParaphrase](https://huggingface.co/datasets/csebuetnlp/BanglaParaphrase) dataset. This is a sequence to sequence transformer model pretrained with the ["Span Corruption"]() objective. Finetuned models using this checkpoint achieve    competitive results on the dataset.
  
  For finetuning and inference, refer to the scripts in the official GitHub repository of [BanglaNLG](https://github.com/csebuetnlp/BanglaNLG).
  
  **Note**: This model was pretrained using a specific normalization pipeline available [here](https://github.com/csebuetnlp/normalizer). All finetuning scripts in the official GitHub repository use this normalization by default. If you need to adapt the pretrained model for a different task make sure the text units are normalized using this pipeline before tokenizing to get best results. A basic example is given below:
  
  ## Using this model in `transformers`
  
  ```python
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
  from normalizer import normalize # pip install git+https://github.com/csebuetnlp/normalizer
  
  model = AutoModelForSeq2SeqLM.from_pretrained("csebuetnlp/banglat5_banglaparaphrase")
  tokenizer = AutoTokenizer.from_pretrained("csebuetnlp/banglat5_banglaparaphrase", use_fast=False)
  
  input_sentence = ""
  input_ids = tokenizer(normalize(input_sentence), return_tensors="pt").input_ids
  generated_tokens = model.generate(input_ids)
  decoded_tokens = tokenizer.batch_decode(generated_tokens)[0]
  
  print(decoded_tokens)
  ```
  
  ## Benchmarks
   
  * Supervised fine-tuning
  
  | Test Set | Model | sacreBLEU | ROUGE-L | PINC | BERTScore | BERT-iBLEU |
  | -------- | ----- | --------- | ------- | ---- | --------- | ---------- |
  | [BanglaParaphrase](https://huggingface.co/datasets/csebuetnlp/BanglaParaphrase) | [BanglaT5](https://huggingface.co/csebuetnlp/banglat5)<br>[IndicBART](https://huggingface.co/ai4bharat/IndicBART)<br>[IndicBARTSS](https://huggingface.co/ai4bharat/IndicBARTSS)| 32.8<br>5.60<br>4.90 | 63.58<br>35.61<br>33.66 | 74.40<br>80.26<br>82.10 | 94.80<br>91.50<br>91.10 | 92.18<br>91.16<br>90.95 |
  | [IndicParaphrase](https://huggingface.co/datasets/ai4bharat/IndicParaphrase) |BanglaT5<br>IndicBART<br>IndicBARTSS| 11.0<br>12.0<br>10.7| 19.99<br>21.58<br>20.59| 74.50<br>76.83<br>77.60| 94.80<br>93.30<br>93.10 | 87.738<br>90.65<br>90.54|
  
  
  The dataset can be found in the link below:
  * **[BanglaParaphrase](https://huggingface.co/datasets/csebuetnlp/BanglaParaphrase)**
  
  ## Citation
  
  If you use this model, please cite the following paper:
  ```
  @article{akil2022banglaparaphrase,
    title={BanglaParaphrase: A High-Quality Bangla Paraphrase Dataset},
    author={Akil, Ajwad and Sultana, Najrin and Bhattacharjee, Abhik and Shahriyar, Rifat},
    journal={arXiv preprint arXiv:2210.05109},
    year={2022}
  }
  ```