readme: # Tokenizer
  
  We trained our tokenizer using [sentencepiece](https://github.com/google/sentencepiece)'s unigram tokenizer. Then loaded the tokenizer as MT5TokenizerFast.
  
  ## Model
  
  We used [MT5-base](https://huggingface.co/google/mt5-base) model.
  
  ## Datasets
  
  We used [Code Search Net](https://huggingface.co/datasets/code_search_net)'s dataset and some scrapped data from internet to train the model. We maintained a list of datasets where each dataset had codes of same language.
  
  ## Plots
  
  ### Train loss
  
  ![train loss](https://i.ibb.co/x53Wm8n/train-loss.png)
  
  ### Evaluation loss
  
  ![eval loss](https://i.ibb.co/McB2jnf/eval-loss.png)
  
  ### Evaluation accuracy
  
  ![eval accuracy](https://i.ibb.co/YDGhLdn/eval-accuracy.png)
  
  ### Learning rate
  
  ![learning rate](https://i.ibb.co/CMStzWv/learning-rate.png)
  
  ## Fine tuning (WIP)
  
  We fine tuned the model with [CodeXGLUE code-to-code-trans dataset](https://huggingface.co/datasets/code_x_glue_cc_code_to_code_trans), and scrapper data.
  