Problems when restoring the pretrain weights for TFbert
## 🐛 Bug

<!-- Important information -->

Model I am using Bert:

Language I am using the model on (English, Chinese....):

The problem arise when using:
* [ ] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

1.when I try to reload the weights for TFbert by using: 
    model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
2. I got the error 
    ValueError: Expected floating point type, got <dtype: 'int32'>.
3. Does anybody know how to fix this?

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS:
* Python version:
* PyTorch version:
* PyTorch Transformers version (or branch):
* Using GPU ?
* Distributed of parallel setup ?
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

Hi, could you provide information about your setup? Which TensorFlow version are you using, which python version, which Transformers version? Thank you.<|||||>> Hi, could you provide information about your setup? Which TensorFlow version are you using, which python version, which Transformers version? Thank you.

Hi, the information about my setup:
Python version: 3.6.9
Tensorflow Version: 2.0.0b1
PyTorch Transformers version (or branch):2.1.1

Thank you<|||||>I can't reproduce this on my side while on the same versions. Would you happen to have a folder called "bert-base-uncased" in the same directory from which you're calling the python script?<|||||>Hi,

I have exactly the same error while doing the same task...

Python version: Python 3.5.3
Tensorflow Version: 2.0.0-beta1
PyTorch Transformers version : transformers (2.1.1)
<|||||>In my environment, **it works as expected**!
@YumingLiu1996 @LeonardoGracioS 

N.B: It works without `force_download=True` parameter too.

```
>>> from transformers import TFBertForSequenceClassification
2019-11-12 14:38:24.462663: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-11-12 14:38:24.466451: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3408000000 Hz
2019-11-12 14:38:24.466942: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56248b3c66f0 executing computations on platform Host. Devices:
2019-11-12 14:38:24.466956: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
>>> model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased', force_download=True)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 313/313 [00:00<00:00, 154303.85B/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 536063208/536063208 [00:48<00:00, 11140340.62B/s]
>>> import platform
>>> platform.platform()
'Linux-4.15.0-66-generic-x86_64-with-debian-buster-sid'
>>> platform.python_version()
'3.6.9'
>>> import torch
>>> torch.__version__
'1.0.1'
>>> import tensorflow as tf
>>> tf.__version__
'2.0.0'
>>> import transformers
>>> transformers.__version__
'2.1.1'
```<|||||>> I can't reproduce this on my side while on the same versions. Would you happen to have a folder called "bert-base-uncased" in the same directory from which you're calling the python script?



> I can't reproduce this on my side while on the same versions. Would you happen to have a folder called "bert-base-uncased" in the same directory from which you're calling the python script?

No, I don't have such as folder.. <|||||>> I can't reproduce this on my side while on the same versions. Would you happen to have a folder called "bert-base-uncased" in the same directory from which you're calling the python script?

I redownload tensorflow 2.0.0 and transformers. And now I encounter another problem:

```
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5f.pyx in h5py.h5f.open()

OSError: Unable to open file (truncated file: eof = 485070229, sblock->base_addr = 0, stored_eof = 497933648)
```
Do you know the solution for this problem?
<|||||>@YumingLiu1996 the error `OSError: Unable to open file`  tells you that the file was not downloaded in its entirety. Please add the `force_download=True` parameter when downloading the pre-trained model:

```py
model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased', force_download=True)
```<|||||>I solved the issue by using a Google Cloud Platform virtual machine with a TF 2.0 environment.<|||||>> I solved the issue by using a Google Cloud Platform virtual machine with a TF 2.0 environment.

I am using a Google Cloud Platform with an K80 and I have the TF2 installed but it doesn't works...
the same problem here guys. 

I don't know why, but it only works on Google Colab until this comment. <|||||>> > I solved the issue by using a Google Cloud Platform virtual machine with a TF 2.0 environment.
> 
> I am using a Google Cloud Platform with an K80 and I have the TF2 installed but it doesn't works...
> the same problem here guys.
> 
> I don't know why, but it only works on Google Colab until this comment.

Do you have exactly the same error ? Did you start your virtual machine with a TF 2.0 environment and not just a "pip install tensorflow=2.0.0" ?<|||||>I did through the pip install, because I have this machine running some
other things. So, the problem is the same but I think could be the
Tensorflow version.

I will try to build TF 2 from source.



Em sex, 15 de nov de 2019 06:07, Samuel Leonardo Gracio <
notifications@github.com> escreveu:

> I solved the issue by using a Google Cloud Platform virtual machine with a
> TF 2.0 environment.
>
> I am using a Google Cloud Platform with an K80 and I have the TF2
> installed but it doesn't works...
> the same problem here guys.
>
> I don't know why, but it only works on Google Colab until this comment.
>
> Do you have exactly the same error ? Did you start your virtual machine
> with a TF 2.0 environment and not just a "pip install tensorflow=2.0.0" ?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/1780?email_source=notifications&email_token=AB4RK3ZWJWXUQDPECFSVTPLQTZRGXA5CNFSM4JLK64N2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEEZC4Q#issuecomment-554275186>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB4RK32QNPOWOL3OVKAP2FTQTZRGXANCNFSM4JLK64NQ>
> .
>
<|||||>same problem here:

    OS:
    Python version: 3.6.8
    PyTorch version: 1.2.0
    TF version: 2.0.0b1
    PyTorch Transformers version (or branch): master (Nov 25, 2019)

model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')

ValueError: Expected floating point type, got <dtype: 'int32'>.

looks like a tensorflow problem
<|||||>I've found your problem! It's a TensorFlow bug in version 2.0.0beta1. If you uninstall this version with `pip uninstall tensorflow` and after that you install the TensorFlow with version 2.0.0a0 with `pip install tensorflow==2.0.0,` **the code works as expected**!

I've tried different settings and it works as expected!
- PyTorch 1.2.0, TensorFlow(CPU-version) 2.0.0
- PyTorch 1.2.0, TensorFlow(GPU-version) 2.0.0
- PyTorch 1.3.1, TensorFlow(CPU-version) 2.0.0
- PyTorch 1.3.1, TensorFlow(GPU-version) 2.0.0

> same problem here:
> 
> ```
> OS:
> Python version: 3.6.8
> PyTorch version: 1.2.0
> TF version: 2.0.0b1
> PyTorch Transformers version (or branch): master (Nov 25, 2019)
> ```
> 
> model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
> 
> ValueError: Expected floating point type, got <dtype: 'int32'>.
> 
> looks like a tensorflow problem<|||||>thanks. I tryied with a0 but it has even more problem, like 

'tensorflow.python.keras.api._v2.keras.layers' has no attribute 'LayerNormalization'

which are said to be bugs in a0 and patched in b1. 

However b1 does not work because
ValueError: Expected floating point type, got <dtype: 'int32'>.

Looks like this is a deadlock.

> I've found your problem! It's a TensorFlow bug in version 2.0.0beta1. If you uninstall this version with `pip uninstall tensorflow` and after that you install the TensorFlow with version 2.0.0a0 with `pip install tensorflow==2.0.0,` **the code works as expected**!
> 
> I've tried different settings and it works as expected!
> 
>     * PyTorch 1.2.0, TensorFlow(CPU-version) 2.0.0
> 
>     * PyTorch 1.2.0, TensorFlow(GPU-version) 2.0.0
> 
>     * PyTorch 1.3.1, TensorFlow(CPU-version) 2.0.0
> 
>     * PyTorch 1.3.1, TensorFlow(GPU-version) 2.0.0
> 
> 
> > same problem here:
> > ```
> > OS:
> > Python version: 3.6.8
> > PyTorch version: 1.2.0
> > TF version: 2.0.0b1
> > PyTorch Transformers version (or branch): master (Nov 25, 2019)
> > ```
> > 
> > 
> > model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
> > ValueError: Expected floating point type, got <dtype: 'int32'>.
> > looks like a tensorflow problem

<|||||>I don't know what to say. I've just tested with Transformers v-2.2.0 and it works as expected. Sorry, but in my environment it works as I've said some days ago. How can i help you? How can i replicate your problem? Which OS are you using?

> thanks. I tryied with a0 but it has even more problem, like
> 
> 'tensorflow.python.keras.api._v2.keras.layers' has no attribute 'LayerNormalization'
> 
> which are said to be bugs in a0 and patched in b1.
> 
> However b1 does not work because
> ValueError: Expected floating point type, got <dtype: 'int32'>.
> 
> Looks like this is a deadlock.
> 
> > I've found your problem! It's a TensorFlow bug in version 2.0.0beta1. If you uninstall this version with `pip uninstall tensorflow` and after that you install the TensorFlow with version 2.0.0a0 with `pip install tensorflow==2.0.0,` **the code works as expected**!
> > I've tried different settings and it works as expected!
> > ```
> > * PyTorch 1.2.0, TensorFlow(CPU-version) 2.0.0
> > 
> > * PyTorch 1.2.0, TensorFlow(GPU-version) 2.0.0
> > 
> > * PyTorch 1.3.1, TensorFlow(CPU-version) 2.0.0
> > 
> > * PyTorch 1.3.1, TensorFlow(GPU-version) 2.0.0
> > ```
> > 
> > 
> > > same problem here:
> > > ```
> > > OS:
> > > Python version: 3.6.8
> > > PyTorch version: 1.2.0
> > > TF version: 2.0.0b1
> > > PyTorch Transformers version (or branch): master (Nov 25, 2019)
> > > ```
> > > 
> > > 
> > > model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
> > > ValueError: Expected floating point type, got <dtype: 'int32'>.
> > > looks like a tensorflow problem<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
