Cannot set deterministic=False for FlaxRobertaPreTrainedModel, therefore dropout doesn't work?
Looking at FlaxRobertaPreTrainedModel's [call function](https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/models/roberta/modeling_flax_roberta.py#L610), there doesn't seem to be a way to set deterministic=False. Therefore, when it calls self.module.apply it leaves deterministic on its default (True). This isn't necessarily a problem if you just want to do inference with the model, but if we want to fine tune then we cannot use dropout. Is there some other place to set this argument that I am missing?
Actually as far as I can tell, the argument train: bool is acting as the deterministic switch. So if I set train=True, then dropout is used. The documentation could use some explanation in my opinion, and also it needs to be explained how if this is true then we need to provide a dropout seed.<|||||>Hi, you are right. `train` is used to control the `deterministic` arg, which is only used in internal modules. All public-facing models expose `train` arg to prevent this. And yes, this should be documented. 

>  also it needs to be explained how if this is true then we need to provide a dropout seed.

The seed is provided when initialising/loading the model, and the default is `True`. And the dropout seed is created by splitting this main seed.

Would you like to open a PR to add the docstring for `train` arg  ? We'll need to update this for all models in the `..._INPUTS_DOCSTRING` variable.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.