Hi @rashub, thanks for sharing the Colab notebook. This seems to be a problem with the fasthugs library, not transformers so perhaps @morgan (the creator) can shed some light on this?
For what it’s worth, I checked that it’s not a fastai problem by feeding your inputs to a ULMFiT classifier from this tutorial: Transfer learning in text | fastai (see screenshot below)
In general the predicted values should not change if you decide to pass a batch instead of a single item.
P.S. you can get nicely formatted code in the forum by surrounding the code snippet with 3 backticks “`”
Screen Shot 2021-02-12 at 5.19.16 pm1562×792 124 KB