How to pass the attention mask as a param to model forward when using torchscript?
## Environment info

- `transformers` version:
- Platform: Ubuntu 16.04
- Python version: 3.6.9
- PyTorch version (GPU): 1.3.0+cu100
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

## Information
 I am using a Bert model downloaded from hugging face. I finetuned that model for a two-class classification task and convert that model to a `torchscript` through the `jit.trace`.



The following code show how I got the torchscript:

    from transformers import BertTokenizer, BertForSequenceClassification
    tokenizer = BertTokenizer.from_pretrained(tokenizer_dir)
    model = BertForSequenceClassification.from_pretrained(model_dir, num_labels=2, torchscript=True)
    model.eval()
    model = model.to("cuda:0")
    
    input_text = ["test this case", "test test this case"] 
    encoding = tokenizer(text_batch, return_tensors='pt', padding=True, truncation=False)
    input_ids = encoding['input_ids']
    attention_mask = encoding['attention_mask']
    
    tokens_tensor = torch.tensor(input_ids)
    tokens_tensor = tokens_tensor.to("cuda:0")
    traced_model = torch.jit.trace(model, tokens_tensor)
    torch.jit.save(traced_model, str(pt_path))


The following code show how I use the torchscript, the input tensor is the same with the first code:

    pt_model = torch.jit.load(model_path))
    pt_model.eval()
    pt_label = pt_model(input_tensor)[0]


For the normal model, it needs to pass two params input tensor and attention mask like:

    model(input, attention_mask=attn_mask)


But for the `torchscript`, I can't pass the attention mask to the model.
So, what's the right way to use `torchscript` to do the forward with the attention mask?

Thanks!



I think you would need to compile your model with both the tokens tensor and the attention mask. Given that the attention mask is the second argument, you can pass it directly when tracing the model:

```py
traced_model = torch.jit.trace(model, [tokens_tensor, attention_mask])
```

then you can do:
```py
model(tokens_tensor, attention_mask)
```<|||||>@LysandreJik It works. Thanks for your help! 