what's the difference between Megatron-gpt2 and GPT2 inside transformers?
what's the difference between Megatron-gpt2 and GPT2 inside transformers?
Hi @jinfagang ! For such general questions please use the [forum](https://discuss.huggingface.co/). We use issues for bug reports and feature requests.

Megatron-gpt2  is the models trained using the megatron library. The arch is similar to the GPT2 arch. You can find more info in the doc https://huggingface.co/docs/transformers/model_doc/megatron_gpt2<|||||>@patil-suraj thanks. I'll move to forum for further discuss. hope you can give me a hand on understanding.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.