Megatron-11B
### Model description

I discovered two implementations of this facebook model on the hub, which was trained on the same corpus as Roberta/Bert. I want to try out some prompting, but when I try to download it with 

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("hyunwoongko/megatron-11B")


I get a 
KeyError: 'megatron' exception. 

This is a relatively sizable model which is as interesting as gpt-j to me, does it work with transformers? 
I found one of the two model re-publishers below wrote their own library, but it depends on outdated versions of transformer. Can we use this model with transformers? 


### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The two megatron-to-pytorch models on the hub: 
https://huggingface.co/models?search=megatron-11

Extra py lib which should make it work: 
https://pypi.org/project/megatron-11b/
I'm still learning prompting, but after poking around a bit with this model using the pylib I shared above, I found the generated text was of quite low quality for the model size. I got way better results with GPT-J and even GPT-2. 
Despite its size, I doubt this model is useful enough. <|||||>When I got coherent sentences out, it switched topics every second sentence. 