Can DistilBert.forward() support token_type_ids ?
I am using DistilBert to distill a pretrained Bert model. That is Bert -> DistilBert.
The input of Bert is a sentence pair: [CLS] Hello word [SEP] Hello Python [SEP].
But DistilBert dose not support sentence pair inputs.
Can DistilBert support sentence pair-like inputs?
DistilBERT can support sentence pair-like inputs but does not make use of token type IDs. It detects sentence pairs according to the special tokens. cc @VictorSanh <|||||>@Yusifu Did you find a solution for this problem? I'm also doing sentence-pair classification (NLI) with Distilbert.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
