I have a similar doubt as regards point 2.
I am working on Question Answering with Distilbert.
The predict function in the Trainer does not work if output_hidden_states = True. It works fine if the same argument is set to False.
Is this a bug? If not, then how  is one to use a model for prediction if one has set the argument output_hidden_states = True while initializing the model ?