I think that these models could be still finetuned on BoolQ style QA datasets :

  

      paperswithcode.com
  

  
    

Papers with Code - BoolQ Benchmark (Question Answering) 4

  The current state-of-the-art on BoolQ is T5-11B. See a full comparison of 7 papers with code.