CausalLM vs HeadModel
@patrickvonplaten, @LysandreJik @sgugger

GPT-Neo implements the class `GPTNeoForCausalLM` and GPT-2 implements the class `GPT2LMHeadModel`. These look like they're supposed to do roughly the same thing. What is the reasoning behind having different names? Do they have any functional differences (other than using different models obviously)?
They are exactly the same `LMHeadModel` was a badly chosen name in the beginning of the library - we are trying to have all causal language models called `...ForCausalLM` now<|||||>> They are exactly the same `LMHeadModel` was a badly chosen name in the beginning of the library - we are trying to have all causal language models called `...ForCausalLM` now

I see. FYI, there are already downstream users who are basing their codebases on `LMHeadModel` such as [Google's BIG-Bench](https://github.com/google/BIG-bench/blob/main/bigbench/models/huggingface_models.py). I worry that this disconnect will build significant technical debt if it is not resolved promptly.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.