support no gpt_j_residual for gpt-neox
# What does this PR do?

Support "gpt_j_residual == False"  for gpt-neox, which implement "else" branch in https://github.com/EleutherAI/gpt-neox/blob/main/megatron/model/transformer.py#L627


## Who can review?
Anyone in the community is free to review the PR.
@sgugger @patrickvonplaten 
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18089). All of your documentation changes will be reflected on that endpoint.<|||||>@TopIdiot, would existing checkpoints (i.e., https://huggingface.co/EleutherAI/gpt-neox-20b) be usable with this configuration option, or would they output gibberish?<|||||>> @TopIdiot, would existing checkpoints (i.e., https://huggingface.co/EleutherAI/gpt-neox-20b) be usable with this configuration option, or would they output gibberish?

@LysandreJik  Yes, it can be used. The default value for gpt_j_residual is "True",  so the exsiting checkpoints would enter the "if self.gpt_j_residual" branch. btw, I tested gpt-neox-20b, the result looks good for me.<|||||>The question is whether the model would give sensible results if you set it to `False`. Transformers is not a modular toolbox, we don't add options to models that make no sense with the pretrained checkpoints of that model, we add new models instead.<|||||>> The question is whether the model would give sensible results if you set it to `False`. Transformers is not a modular toolbox, we don't add options to models that make no sense with the pretrained checkpoints of that model, we add new models instead.

@sgugger gpt_j_residual  is an option which already existed in gpt-neox (i.e.  https://github.com/EleutherAI/gpt-neox/blob/main/configs/20B.yml#L29)

Recently, we trained EleutherAI/gpt-neox with option gpt_j_residual == False and successfully convert the checkpoint to torch. However, we found that the current version of huggingface's gpt-neox doesn't implement this part. We added it by ourself and got expected result. <|||||>You can share the custom code of your model using the [code in the Hub](https://huggingface.co/docs/transformers/custom_models) API. This is typically the kind of changes we don't accept in existing model files (and the reason there is one for GPT-2, GPT-J, GPT-Neo and GPT-Neo-X which are all very similar).<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.