gptj doc tied weights mistake?
https://github.com/huggingface/transformers/blob/5f1918a4a8ed893822aa7dd2b75acf83f255ad79/src/transformers/models/gptj/modeling_gptj.py#L686
```
    The GPT-J Model transformer with a language modeling head on top (linear layer with weights tied to the input
    embeddings).
```

The lm head isn't tied to the input embeddings for gpt-j
Good catch!