Environment info


transformers version: 3.5.1
Platform: Linux-4.14.203-116.332.amzn1.x86_64-x86_64-with-glibc2.10
Python version: 3.7.6
PyTorch version (GPU?): 1.7.0 (True)
Tensorflow version (GPU?): 2.3.1 (True)
Using GPU in script?: No
Using distributed or parallel set-up in script?: No

Information
Model I am using (Bert, XLNet …): Bert
The problem arises when using:

my own modified scripts: (give details below)

The tasks I am working on is:

my own task or dataset: (give details below)

To reproduce
Steps to reproduce the behavior:

Trained HuggingFace Transformers model BertForSequenceClassification on custom dataset with PyTorch backend.
Used provided convert_graph_to_onnx.py script to convert model (from saved checkpoint) to ONNX format.
Loaded the model with ONNXRuntime
Instantiated BertTokenizer.from_pretrained(‘bert-based-uncased’) and fed in various input text to encode_plus method.
Fed outputs of this to the ONNXRuntime session.

Expected behavior
The expected behavior is that the output of sess.run on the aforementioned inputs should output an array of dimension (1, 100) (corresponding to 100 classes) with each value between 0 and 1, with all entries summing to 1. We get the correct dimension, however, we get values between about -3.04 and 7.14 (unsure what these values refer to).