replacing apex.normalization.FusedLayerNorm with torch.nn.LayerNorm
It seems that time has arrived to drop `apex.normalization.FusedLayerNorm` in favor of `torch.nn.LayerNorm` 

1. the latter was ported more than a year ago from apex  https://github.com/pytorch/pytorch/pull/27634 (around pt-1.4)
2. it's faster than the apex according to my benchmarks https://github.com/pytorch/pytorch/issues/37713#issuecomment-753434842 (**33% faster on rtx-3090!**, 10% faster on gtx-1070)

**but note:** this same benchmark run here https://github.com/pytorch/fairseq/issues/2012#issuecomment-622607286 on V100 reports the opposite  - that the native is slower (pt-1.5). So it might help to run this very quick benchmark on other cards and compare. In particular if you have access to V100 please report back the findings at this thread: https://github.com/pytorch/pytorch/issues/37713

The main reason for this need is that `apex.normalization.FusedLayerNorm` is buggy (corrupts memory) when it comes to switching devices, which is done a lot under Model Parallel. https://github.com/NVIDIA/apex/issues/1022
With  `apex.normalization.FusedLayerNorm` things fail a lot under MP and requires sticking `torch.cuda.set_device(id)` in many many places as a workaround :( Since this overload is used at model's init time it's not possible to not use it under MP as the latter gets activate after model's init.

I will use that workaround if you find out that apex is faster still on some important-to-consider hardware. And, of course, in that case please report back to the pytorch team so that they could fix it. Otherwise apex support is pretty much no more and it's just a matter of time before apex will be unusable.

The models that need that change are bart/fsmt/prophetnet.

@patrickvonplaten, @LysandreJik

I'm good with changing to `torch.nn.LayerNorm`. At @stas00 - do you know what the advantage of `apex.normalization.FusedLayerNorm` is supposed to be? Why did we add `apex.normalization.FusedLayerNorm` in the first place?<|||||>Prior to about a year ago, `apex.normalization.FusedLayerNorm` was faster than `torch.nn.LayerNorm`, but then the former got ported to native `torch.nn.LayerNorm`, and now the native appears to be faster - at least the 2 cards I have experimented with. I checked with pt-1.4 .. pt-1.8.dev

If you have other than gtx-1070/rtx-3090 cards which I benchmarked with please run that benchmark and see if it stands true for other cards: https://github.com/pytorch/pytorch/issues/37713#issuecomment-753434842
it only takes a few seconds if you have apex installed already. To install apex:

```
git clone https://github.com/NVIDIA/apex
cd apex
rm -rf build
pip install  --global-option="--cpp_ext" --global-option="--cuda_ext"  .
```

The benchmark measures and reports a total run time, so the smaller the numbers the faster it is.

If you do run the benchmarks please post your results at https://github.com/pytorch/pytorch/issues/37713 so that it can be seen whether it's safe to drop `apex.normalization.FusedLayerNorm` based on hard data and not anecdotal info.

Thank you.