Support longer sequences with BertForSequenceClassification
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

I am using `BertForSequenceClassification` for solving a regression task. I have a long sequence as an input and the model outputs a float in range [0,1].

Most of my sequences are longer than 512, which is the max sequence length in the current bert pretrained models.

To handle longer sequence you need to split the input with some stride value, As suggested here:
https://github.com/google-research/bert/issues/27#issuecomment-435265194

It seems that it was implemented in the `BertForQuestionAnswering` and the `Squad` example but not in the `BertForSequenceClassification` which i use.
But still, I do not understand how that would really work. I do not understand the use of `start_positions` and `end_positions` enough to implement it on my own.

Given my regression task how do i handle the output of the model for each chunk of my input and get unified output of the model?



A little question because we are trying to organize the issues better:
- what made you not use the issue templates we have added?<|||||>> A little question because we are trying to organize the issues better:
> 
> * what made you not use the issue templates we have added?

Didn't know about it...<|||||>@eladbitton Can you link lines of code to where this was done for `BertForQuestionAnswering` and `SQuAD`? I'd be willing to take a stab at implementation.<|||||>Hey @maxzzze. I was looking at:
https://github.com/huggingface/pytorch-transformers/blob/0d1dad6d5323cf627cb8d7ddd428856ab8475f6b/pytorch_transformers/modeling_bert.py#L1112

Now that i look at it, i am not sure if they implemented it there.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Same issue here.
Since this package is presented as a "plug n play" solution for this kind of task, that is really unfortunate.<|||||>@eladbitton, I believe the start and end positions in BertForQuestionAnswering are for filtering tokens when computing the loss (since the loss is given by the cross-entropy between the predicted and true distributions of the start token, the latter of which is a one-hot vector; similarly for the end token), not for converting a large sequence into a batch of shorter sequences.

@thomwolf, are there plans to add the functionality mentioned [here](https://github.com/google-research/bert/issues/27#issuecomment-435265194) by Devlin (or would you be able to suggest any alternatives that might work)?