Hi @ahotrod, thanks for that hint! I will try to fine-tune the model and report back results here in a few days 
@BramVanroy: for token classification I just use the shell script from here: https://github.com/huggingface/transformers/blob/master/examples/token-classification/run.sh 18
For SQuAD 1 I used the command specified here:

  
      
      GitHub
  
  
    

huggingface/transformers 29

🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0. - huggingface/transformers


  
  
    
    
  
  


A100 would be super interesting