I want to create an executable version of Tokenizer for Bert - Below is a small code piece:
from transformers import AutoTokenizer, AutoModel
import torch

sentences = ['This framework generates embeddings for each input sentence']
tokenizer_model = AutoTokenizer.from_pretrained("sentence-transformers/paraphrase-mpnet-base-v2", torchscript=True)
encoded_input = tokenizer_model(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt')

# !!! complains that 'tokenizer_model' doesn't have eval()
tokenizer_model.eval();    

# !!! tokenizer_model takes list of sentences as inputs, how should I provide tensorial dummpy inputs?
traced_tokenizer_model = torch.jit.trace(tokenizer_model, dummpy_inputs)
torch.jit.save(traced_tokenizer_model, "traced_tokenize_bert.pt")

My first problem is that tokenizer_model doesn’t have eval()
My second problem is that the tokenizer_model takes as inputs a list of strings. How am I supposed to provide tensorial form dummy inputs to create the traced_tokenizer_model?