Automatically allocates memory in GPU, always OOM when create TFALBERT model
## 🐛 Bug

<!-- Important information -->

Model I am using :ALBERT

Language I am using the model on (English, Chinese....):English

> from transformers import TFAlbertModel

> model2=TFAlbertModel.from_pretrained('albert-base-v1')

Then:

> ---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-2-a440a0748e94> in <module>
----> 1 model2=TFAlbertModel.from_pretrained('albert-base-v1')

~/anaconda3/lib/python3.7/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    287         # 'by_name' allow us to do transfer learning by skipping/adding layers
    288         # see https://github.com/tensorflow/tensorflow/blob/00fad90125b18b80fe054de1055770cfb8fe4ba3/tensorflow/python/keras/engine/network.py#L1339-L1357
--> 289         model.load_weights(resolved_archive_file, by_name=True)
    290 
    291         ret = model(model.dummy_inputs, training=False)  # Make sure restore ops are run

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in load_weights(self, filepath, by_name)
    179         raise ValueError('Load weights is not yet supported with TPUStrategy '
    180                          'with steps_per_run greater than 1.')
--> 181     return super(Model, self).load_weights(filepath, by_name)
    182 
    183   @trackable.no_automatic_dependency_tracking

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in load_weights(self, filepath, by_name)
   1169           'first, then load the weights.')
   1170     self._assert_weights_created()
-> 1171     with h5py.File(filepath, 'r') as f:
   1172       if 'layer_names' not in f.attrs and 'model_weights' in f:
   1173         f = f['model_weights']

~/anaconda3/lib/python3.7/site-packages/h5py/_hl/files.py in __init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, **kwds)
    392                 fid = make_fid(name, mode, userblock_size,
    393                                fapl, fcpl=make_fcpl(track_order=track_order),
--> 394                                swmr=swmr)
    395 
    396             if swmr_support:

~/anaconda3/lib/python3.7/site-packages/h5py/_hl/files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    168         if swmr and swmr_support:
    169             flags |= h5f.ACC_SWMR_READ
--> 170         fid = h5f.open(name, flags, fapl=fapl)
    171     elif mode == 'r+':
    172         fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5f.pyx in h5py.h5f.open()

OSError: Unable to open file (file signature not found)

I tried to trace the GPU state, the memory usage is 15513MiB / 16130MiB, it is obvious that 
when I create a model, it automatically allocates memory in GPU, but when I tried this in colab and use the same TF version, it works well, after creating model, there still are much free memory.
*OS: Linux version 4.9.0-11-amd64
* Python version:3.7
* TF version:TF2.0
* Transformers version (or branch):2.2
* Using GPU ?GPU

## Additional context

<!-- Add any other context about the problem here. -->

What is the batch size you used?<|||||>The same bug occurs with Python 3.6.9, Transformers 2.2.1 (installed with `pip install transformers`), PyTorch 1.3.1 and TensorFlow 2.0.
Stack trace:
```
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import TFAlbertModel
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2019-12-05 09:55:19.006308: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-05 09:55:19.027197: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3408000000 Hz
2019-12-05 09:55:19.027888: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5582d4e6e4c0 executing computations on platform Host. Devices:
2019-12-05 09:55:19.027909: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
>>> model = TFAlbertModel.from_pretrained('albert-base-v1')
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 484/484 [00:00<00:00, 185172.23B/s]
299B [00:00, 131456.70B/s]
2019-12-05 09:55:28.628697: W tensorflow/python/util/util.cc:299] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/transformers/modeling_tf_utils.py", line 289, in from_pretrained
    model.load_weights(resolved_archive_file, by_name=True)
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 181, in load_weights
    return super(Model, self).load_weights(filepath, by_name)
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 1171, in load_weights
    with h5py.File(filepath, 'r') as f:
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/h5py/_hl/files.py", line 408, in __init__
    swmr=swmr)
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
```
If I try to use the PyTorch version of Albert with _albert-base-v1_, it works as expected!
Stack trace:
```
>>> model = AlbertModel.from_pretrained('albert-base-v1')
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47376396/47376396 [00:04<00:00, 10033199.48B/s]
```

Said this, I suspect that the TensorFlow version of Albert is not developed correctly (it misses the config). Is it possible? Now I'm investigating on..

UPDATE 1: I've gone to the Transformers' source code in the [modeling_tf_albert.py](https://github.com/huggingface/transformers/blob/e85855f2c408f65a4aaf5d15baab6ca90fd26050/transformers/) and I've downloaded the .h5 model **correctly** (from [this link](https://s3.amazonaws.com/models.huggingface.co/bert/albert-base-v1-tf_model.h5)). So I suspect there is an internal error that is independent from the download of the .h5 file.

> ##  Bug
> Model I am using :ALBERT
> 
> Language I am using the model on (English, Chinese....):English
> 
> > from transformers import TFAlbertModel
> 
> > model2=TFAlbertModel.from_pretrained('albert-base-v1')
> 
> Then:
> 
> > 
> 
> OSError Traceback (most recent call last)
> in 
> ----> 1 model2=TFAlbertModel.from_pretrained('albert-base-v1')
> 
> ~/anaconda3/lib/python3.7/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
> 287 # 'by_name' allow us to do transfer learning by skipping/adding layers
> 288 # see https://github.com/tensorflow/tensorflow/blob/00fad90125b18b80fe054de1055770cfb8fe4ba3/tensorflow/python/keras/engine/network.py#L1339-L1357
> --> 289 model.load_weights(resolved_archive_file, by_name=True)
> 290
> 291 ret = model(model.dummy_inputs, training=False) # Make sure restore ops are run
> 
> ~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in load_weights(self, filepath, by_name)
> 179 raise ValueError('Load weights is not yet supported with TPUStrategy '
> 180 'with steps_per_run greater than 1.')
> --> 181 return super(Model, self).load_weights(filepath, by_name)
> 182
> 183 @trackable.no_automatic_dependency_tracking
> 
> ~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py in load_weights(self, filepath, by_name)
> 1169 'first, then load the weights.')
> 1170 self._assert_weights_created()
> -> 1171 with h5py.File(filepath, 'r') as f:
> 1172 if 'layer_names' not in f.attrs and 'model_weights' in f:
> 1173 f = f['model_weights']
> 
> ~/anaconda3/lib/python3.7/site-packages/h5py/_hl/files.py in **init**(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, **kwds)
> 392 fid = make_fid(name, mode, userblock_size,
> 393 fapl, fcpl=make_fcpl(track_order=track_order),
> --> 394 swmr=swmr)
> 395
> 396 if swmr_support:
> 
> ~/anaconda3/lib/python3.7/site-packages/h5py/_hl/files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
> 168 if swmr and swmr_support:
> 169 flags |= h5f.ACC_SWMR_READ
> --> 170 fid = h5f.open(name, flags, fapl=fapl)
> 171 elif mode == 'r+':
> 172 fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
> 
> h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
> 
> h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
> 
> h5py/h5f.pyx in h5py.h5f.open()
> 
> OSError: Unable to open file (file signature not found)
> 
> I tried to trace the GPU state, the memory usage is 15513MiB / 16130MiB, it is obvious that
> when I create a model, it automatically allocates memory in GPU, but when I tried this in colab and use the same TF version, it works well, after creating model, there still are much free memory.
> *OS: Linux version 4.9.0-11-amd64
> 
> * Python version:3.7
> * TF version:TF2.0
> * Transformers version (or branch):2.2
> * Using GPU ?GPU
> 
> ## Additional context<|||||>> What is the batch size you used?

I haven't tried to train, I just run one line code to create the model, then problem happened. 

> model2=TFAlbertModel.from_pretrained('albert-base-v1')
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
