Any suggestions?
Also what came in my mind. That TFBertForSequence is using the pooled_output. So the model is finetuned viad this pooled_output.
But instead I could use the cls embedding or the globalaveragepooling of the hiddensequence for finetuning (pass to the classifier layer), right?