readme: ---
  language: ru
  license: apache-2.0
  tags:
  - transformers
  thumbnail: "https://github.com/RussianNLP/RuCoLA/blob/main/logo.png"
  widget:
  - text: "Он решил ту или иную сложную задачу."
  ---
  This is a finetuned version of [RuRoBERTa-large](https://huggingface.co/sberbank-ai/ruRoberta-large) for the task of linguistic acceptability classification on the [RuCoLA](https://rucola-benchmark.com/) benchmark.
  
  The hyperparameters used for finetuning are as follows:
  * 5 training epochs (with early stopping based on validation MCC)
  * Peak learning rate: 1e-5, linear warmup for 10% of total training time
  * Weight decay: 1e-4
  * Batch size: 32
  * Random seed: 5
  * Optimizer: [torch.optim.AdamW](https://pytorch.org/docs/stable/generated/torch.optim.AdamW.html)