Setting a dropout rate is dependent on a number of factors such as the type of data and the goal of the model. Some people have had success setting a dropout rate to 20% (1 in 5) but you will likely have to try and test to find the correct dropout rate for your model. Typically, ml models require at least 10 times more training data than there are model parameters. In your case, it would be recommended to have at least 1.28T examples. I would increase the number of examples first and if that does not work; test out different dropout rates.
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence