Add finetuning task support for GPT-J to 4.27.0.dev
### Feature request

I have tried running fine-tuning tasks including QA, Summarization and text classification with GPT-J. As of now only QA could be made working with a minor hack to use distillbert tokenizer. Not sure if this is best. Other tasks do not work for GPT-J. Any help appreciated.
Thanks

### Motivation

Fine-tuning for LLMs (GPT-J)

### Your contribution

I need to understand the HF transformer code to make any contribution. Will submit PRs if I see possible ways to improve.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.