Thanks for the quick reply!
I’m using
from transformers import GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2", 
                                          cache_dir=cache_dir, 
                                          torchscript=True)

Note I’m using the same tensor to trace the model as for inference



 jonrbates:

torch.tensor([[464, 6842, 1816, 625, 262, 8598, 284, 766]])