Fix inverted conditional in TF common test!
Noticed a rather alarming conditional being backwards in the `test_pt_tf_model_equivalence` common test. This probably resulted in a lot of tests being skipped!
_The documentation is not available anymore as the PR was closed or merged._<|||||>As expected this has raised a few bugs in the cross-test that were silent before - I'll see what I can do in this PR<|||||>Most likely - I'll investigate them all soon!<|||||>Quick summary of the fixes needed:

ESM: `TFEsmForTokenClassification` copied the computation from `TFBertForTokenClassification`, but this has some slightly odd BERT-specific behaviour and doesn't mask -100 in the same way as other models. Replaced it with the loss block from `TFRobertaForTokenClassification` and all tests pass.

GPT2: For model classes that take rank-3 inputs (e.g. `MultipleChoice` or `DoubleHeads`), when `output_hidden_states=True` , inputs have their second two dims flattened internally in the main model stem. This means that the output `hidden_states` are rank 3 `(bsz, seq_len * num_choices, hidden_dim)` and not rank 4 `(bsz, num_choices, seq_len, hidden_dim)`. However, the PT model un-flattens the output for the final `hidden_states`, which means the last hidden state is rank-4, unlike the others which remain rank-3. In the old TF model, all hidden states are rank-3. I modified the TF code to un-flatten the last hidden state in the same way.

HUBERT: Loss computation especially for CTC overflows a lot with the default labels, which creates lots of `inf` values and makes it very hard to compare TF and PT losses. I skipped PT-TF equivalence testing for the losses, but keep it for all non-loss outputs.

Wav2Vec2: Same as HUBERT

XGLM: The PT XGLM model does a weird thing where it shifts labels by 1 and then adds `pad_token_id` as the final label to all samples. I'm not sure this is correct, but I modified the TF code to do the same. It's possible the TF code is the right one here though, in which case we should revert it and change the PT code instead.<|||||>@gante I fixed all the bugs that this surfaced, explained above ^

cc @sgugger for final review too<|||||>Thank you for the fix  @Rocketknight1 ❤️ . And I apologize for the mistake I introduced ...