GPT-NeoX checkpoint conversion?
I found that there are conversion from gpt-neo and gpt-J to transformers. I wonder if the checkpoints trained using gpt-neox https://github.com/EleutherAI/gpt-neox could be loaded inside transformers? I think it's the same architecture as GPT-neo but I could be wrong. Thanks!
This is something we're working on at eleuther in preparation for an upcoming release, and already have some working code for specific configurations, but for other configurations it breaks.

If it would be useful to HF, we can post up the code :) <|||||>@sdtblck This sounds really useful and I'd like to take a look too! I wonder if you could share what configuration it works on? Right now I have a trained model that's using the 2.7B.yml and I'd like to do inference with HF (no training is fine). Thank you so much!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>I wonder if there are any updates on this @sdtblck ?
Any help would be greatly appreciated!

