Happy to help 
I think in your case what you might need is something like
return (loss(logits, labels), outputs) if return_output else loss(logits, labels)

since my understanding is that your loss object is really a loss function and we should be returning a scalar in compute_loss. As a sanity check you can try feeding some inputs to your function to see what the outputs look like, e.g.
trainer.compute_loss(model, inputs, return_outputs=False)

PS the reason we now need return_outputs in the signature is because it’s used in the prediction_step function to get the logits during training: transformers.trainer — transformers 4.3.0 documentation 6