HF <-> megatron checkpoint reshaping and conversion for GPT
# What does this PR do? 
With respect to GPT model,
1. Now, user would be able to convert Megatron-LM GPT model having different tensor parallel and pipeline parallel sizes to a universal transformers checkpoint for `gpt2` model. This checkpoint is also sharded by default (10GB per shard) or the value which user provides. Sample command to run to convert from Megatron-LM to Transformers checkpoint (below command is run for a checkpoint having tp-size 2 and pp-size 1):
```
python checkpoint_reshaping_and_interoperability.py \
--convert_checkpoint_from_megatron_to_transformers \
--load_path "megatron_lm_gpt/iter_0005000" \
--save_path "hf_checkpoint" \
--max_shard_size "200MB" \
--tokenizer_name "/home/sourab/code-parrot-minimal" \
--print-checkpoint-structure
```
Ouput logs: https://gist.github.com/pacman100/eedce29f084f3efdac76456bd407f978#file-megatron_to_trfs-log

2. Reverse conversion from transformers checkpoint to Megatron checkpoint with variable TP and PP sizes is supported. Sample command for it is given below (converting to a checkpoint with `target_tensor_model_parallel_size`=2 and `target_pipeline_model_parallel_size`=1. `target_data_parallel_size` is used when `--use_distribured_optimizer` is passed):
```
python checkpoint_reshaping_and_interoperability.py \
--load_path "hf_checkpoint" \
--save_path "megatron_lm_checkpoint" \
--target_tensor_model_parallel_size 2 \
--target_pipeline_model_parallel_size 1 \
--target_data_parallel_size 2 \
--target_params_dtype "bf16" \
--make_vocab_size_divisible_by 128 \
--print-checkpoint-structure 
```
Output logs: https://gist.github.com/pacman100/eedce29f084f3efdac76456bd407f978#file-trfs_to_megatron-log

A quick test to make sure everything is working properly:
Code as well as output logs: https://gist.github.com/pacman100/eedce29f084f3efdac76456bd407f978#file-gistfile1-txt

_The documentation is not available anymore as the PR was closed or merged._<|||||>I can't use this from  transformers to megatron  for accelerate-megatron-plugin to continue finetune the checkpoint. is there any thing i missing? then load the converted checkpoint with resume_from_checkpoint args, it seems that the gpus are reinit and only one gpu is 100% utility. <|||||>Hello, thank you for the converting tools.
By the way, is there any plan that the script of T5 HF model <-> Megatron model will be supported and when?