clean up PT <=> TF conversion
Cleaning up PT <=> TF conversion method.

cc @VictorSanh 
Cool!<|||||># [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=h1) Report
> Merging [#2069](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ee53de7aac8312140e87d452718e15e3d42e27dd?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2069/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #2069   +/-   ##
=======================================
  Coverage   83.16%   83.16%           
=======================================
  Files         109      109           
  Lines       15858    15858           
=======================================
  Hits        13188    13188           
  Misses       2670     2670
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2069/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3V0aWxzLnB5) | `90.86% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=footer). Last update [ee53de7...1d87b37](https://codecov.io/gh/huggingface/transformers/pull/2069?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>@thomwolf and @VictorSanh - could you please look into #2088 (PT to TF)? tagging you guys in this thread as it seems relevant, please let me know otherwise. 

Thanks<|||||>This thread is not relevant but I'll give a look at your issue soon.<|||||>thanks!