problem about change from pytorch-pretrained-bert to transformers
I am doing the sentiment analysis with pytorch-pretrained-bert, it work correct, but when I change to transformers, the accuracy on the dev data is poor (but the accuracy with pytorch-pretrained-bert is quite higher). Besides, I notice the Migrating from pytorch-pretrained-bert to transformers module.
the input is:
[CLS] sentence [SEP] aspect [SEP]
and just using BertForSequenceClassification (also try add an classifier after BertModel), but all fail on transformers.
Can Sombody figure out what's wrong with it? Thanks!
OK, the problem is, in transformers, forward() method args position is little bit different compare to pytorch-pretrained-bert. the order change from (token, segment, mask) to (token, mask, segment). So pass the args by name will always right, lol