Solved by AlekseyDorkin in post #2 
              
              
                This kind of problem is a task in the SuperGLUE benchmark. 
Generally, the approach is to fine-tune a BERT-like model with question/context pairs with a SEP token (or equivalent) separating them. Accordingly, labels correspond to yes/no answers. 
You can use BERT for sequence classification model to…