Hi,
Looking at your code, you can already make it faster in two ways: by (1) batching the sentences and (2) by using a GPU, indeed.
Deep learning models are always trained in batches of examples, hence you can also use them at inference time on batches. The tokenizer also supports preparing several examples at a time.
Here’s a code example:
from transformers import BertTokenizer, BertForSequenceClassification
import torch

model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name)

sentences = ["I like this movie a lot", "This movie is super bad"]
encoding = tokenizer(sentences, max_length=512, truncation=True, padding='max_length', return_tensors='pt')

# set device to GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# put model on GPU (is done in-place)
model.to(device)

# put data on GPU
for k,v in encoding.items():
     encoding[k] = v.to(device)

# forward pass
outputs = model(**encoding)

# get predicted class indices
predicted_class_indices = outputs.logits.argmax(-1).tolist()
# turn into actual class names
predicted_classes = [model.config.id2label[label] for label in predicted_class_indices]
print(predicted_classes)

If you want to speed it up even more, then you can indeed look at converting your trained model to ONNX.