Let's have a look inside the question answering pipeline. The question answering pipeline can extracts answers to questions from a given context or passage of text, like this part of the Transformers repo README. It also works for very long contexts, even if the answer is at the very end, like in this example. In this video, we will see why! The question answering pipeline follows the same steps as the other pipelines: the question and context are tokenized as a sentence pair, fed to the model then some post-processing is applied. The tokenization and model steps should be familiar. We use the auto class suitable for Question Answering instead of sequence classification, but one key difference with text classification is that our model outputs two tensors named start logits and end logits. Why is that? Well this is the way the model finds the answer to the question. First let's have a look at the model inputs. It's numbers associated with the tokenization of the question followed by the context (with the usual CLS and SEP special tokens). The answer is a part of those tokens. So we ask the model to predict which token starts the answer and which ends the answer. For our two logit outputs, the theoretical labels are the pink and purple vectors. To convert those logits into probabilities, we will need to apply a SoftMax, like in the text classification pipeline. We just mask the tokens that are not part of the context before doing that, leaving the initial CLS token unmasked as we use it to predict an impossible answer. This is what it looks in terms of code. We use a large negative number for the masking, since its exponential will then be 0. Now the probability for each start and end position corresponding to a possible answer, we give a score that is the product of the start probabilities and end probabilities at those positions. Of course, a start index greater than an end index corresponds to an impossible answer. Here is the code to find the best score for a possible answer. Once we have the start and end positions of the tokens, we use the offset mappings provided by our tokenizer to find the span of characters in the initial context, and get our answer! Now, when the context is long, it might get truncated by the tokenizer. This might result in part of the answer, or worse, the whole answer, being truncated. So we don't discard the truncated tokens but build new features with them. Each of those features contains the question, then a chunk of text in the context. If we take disjoint chunks of texts, we might end up with the answer being split between two features. So instead, we take overlapping chunks of texts, to make sure at least one of the chunks will fully contain the answer to the question. The tokenizers do all of this for us automatically with the return overflowing tokens option. The stride argument controls the number of overlapping tokens. Here is how our very long context gets truncated in two features with some overlap. By applying the same post-processing we saw before for each feature, we get the answer with a score for each of them, and we take the answer with the best score as a final solution.