3 sentences as input for BertForSequenceClassification?
Hi there,

Thanks for releasing this awesome repo, it does lots people like me a great favor.

So far I've tried sentence-pair BertForSequenceClassification task, and it indeed work. I'd like to know if it is possible to use BertForSequenceClassification to model triple sentences classification problem and its input can be described as below:

    **[CLS]A[SEP]B[SEP]C[SEP]**

Expecting for your reply!

Thanks & Regards
Technically it is possible but BERT was not pretrained to handle multiple SEP tokens between sentences and does not have a third token_type, so I think it won't be easy to make it work. You may also want to use a new token for the second separation.<|||||>> Technically it is possible but BERT was not pretrained to handle multiple SEP tokens between sentences and does not have a third token_type, so I think it won't be easy to make it work. You may also want to use a new token for the second separation.

Hi artemisart,

Thanks for your reply.

So, if someone wanna take multiple sentences as input of BertForSequenceClassification, let's say a whole passage, an alternative way is to concatenate them into a single "sentence" and then fit it in, right?<|||||>I you don't have a separation (like question/answer) then yes you can just concatenate them (but you are still limited to 512 tokens).<|||||>@mikelkl I would also go with the solution and answer of @artemisart.<|||||>@artemisart  hi, if i have a single sentence classification task, should the max length of sentence limited to half of 512,  that is to say 256?<|||||>No, it will be better if you use the full 512 tokens.<|||||>wouldn't concatenating the whole passage into a single sentence mean losing context of each sentence? @artemisart <|||||>No it shouldn't <|||||>What if I want to check on a huge corpus, that even concatenating into one sentence exceeds the 512 token limit? @artemisart<|||||>@thedrowsywinger maybe u should try Transformer-XL<|||||>> I you don't have a separation (like question/answer) then yes you can just concatenate them (but you are still limited to 512 tokens).

I have 3 inputs, 1 of the input contains conversation (QUERY, ANSWER). 
QUERY: I want to ask a question.

> ANSWER: Sure, ask away.
> QUERY: How is the weather today?
> ANSWER: It is nice and sunny.
> QUERY: Okay, nice to know.
> ANSWER: Would you like to know anything else?

How can I tell the model to separate the turns of conversation? Model is classification model.
I was thinking to add a new special token <EOT> between the turns but could not get it work.