Fix CodeParrot training script
This PR fixes some features in the training script of CodeParrot:

* use `Pytorch` implementation of `AdamW` instead of `transformers` implementation
* add shuffling of the sequences in the batches
* fix error in weight decay for LayerNorm
* change the tracked loss to the average over batches instead of the main worker loss + compute average over accumulated steps manually for wandb/tensorboard plot
* update requirements

cc @lvwerra
_The documentation is not available anymore as the PR was closed or merged._