Mobilenet_v1 Dropout probability
Hi @amyeroberts [here](https://github.com/huggingface/transformers/pull/17799) is the pull request with some interesting bug. Mobilenetv2 made from tensorflow checkpoint, but in Tensorflow dropout prob is the fraction of **non zero** values, but in PyTorch this is fraction of **zero** values. It should be **1 - prob** for PyTorch.

Load Mobilenetv1 model from 'google/mobilenet_v1_1.0_224'

Dropout prob should be 1 - tf_prob
@andreysher Thanks for raising this issue. The dropout rate `p` or `rate` is defined the same for TensorFlow and PyTorch layers. 

From the [TensorFlow documentation](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout): 
> The Dropout layer randomly sets input units to 0 with a frequency of `rate` at each step during training time

From the [PyTorch documentation](https://pytorch.org/docs/stable/generated/torch.nn.Dropout.html): 
> During training, randomly zeroes some of the elements of the input tensor with probability `p` 

Is there something specific about the original MobileNet implementation or checkpoints which means this doesn't apply? 
<|||||>Thanks for this clarification! But i don't understand why default dropout_prob in the PR is 0.999? If i load model by 
```
from transformers import AutoImageProcessor, MobileNetV1ForImageClassification

model = MobileNetV1ForImageClassification.from_pretrained("google/mobilenet_v1_1.0_224")
```
i get model with dropout p = 0.999. This is unexpected. Could you enplane why such value is presented in pretrained model?<|||||>@andreysher Ah, OK, I think I see what the issue is. The reason the dropout value p=0.999 is because it's set in the [model config](https://huggingface.co/google/mobilenet_v1_1.0_224/blob/dd9bc45ff57d9492e00d48547587baf03f0cdade/config.json#L5). This is a mistake as, as you had identified, the probability in the original mobilenet repo is the [keep probability](https://github.com/tensorflow/models/blob/ad32e81e31232675319d5572b78bc196216fd52e/research/slim/nets/mobilenet_v1.py#L306) i.e. `(1 - p)`. I've opened PRs on the hub to update the model configs:

- https://huggingface.co/google/mobilenet_v1_0.75_192/discussions/2
- https://huggingface.co/google/mobilenet_v2_1.0_224/discussions/3
- https://huggingface.co/google/mobilenet_v1_1.0_224/discussions/2
- https://huggingface.co/google/mobilenet_v2_0.35_96/discussions/2
- https://huggingface.co/google/mobilenet_v2_1.4_224/discussions/2
- https://huggingface.co/google/mobilenet_v2_0.75_160/discussions/2

Thanks for flagging and the time take to explain. <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.