Hi @hhschu,
Thank you for the information provided with regards to this model, it is incredibly useful. I also noticed that the scores given with this model were lower than for other models, which is odd.
I will try to use the fine-tuned MobileBERT then and see whether the results improve. What accuracy did you obtain from fine-tuning MobileBERT on MNLI and MNLI-mm, if that is alright to ask?