BertForQuestionAnswering: Predicting span on the question?
Hello,

I have a question regarding the `BertForQuestionAnswering` implementation. If I am not mistaken, for this model the sequence should be of the form `Question tokens [SEP] Passage tokens`. Therefore, the embedded representation computed by `BertModel` returns the states of both the question and the passage (a tensor of length `passage + question + 1`).

If I am not mistaken, the span logits are then calculated for the whole sequence, i.e. **they can be calculated for the question** even if the answer is always in the passage (see [the model code](https://github.com/huggingface/pytorch-pretrained-BERT/blob/8da280ebbeca5ebd7561fd05af78c65df9161f92/pytorch_pretrained_bert/modeling.py#L1097) and the [squad script](https://github.com/huggingface/pytorch-pretrained-BERT/blob/8da280ebbeca5ebd7561fd05af78c65df9161f92/examples/run_squad.py#L899)). I wonder if this behavior is really desirable. Doesn't it confuse the model?

Thank you for your work!
This is the original behavior from the TF implementation.
The predictions are filtered afterward (in `write_predictions`) so this is probably not a big issue.
Maybe try with another behavior and see if it improve upon the results?