Dataset and DataCollator for BERT Next Sentence Prediction (NSP) task.
I am working with some pre-training tasks recently, and found that transformers have not yet supported NSP tasks. 
So I write the code of **Dataset and DataCollator for BERT NSP task**, now I try to contribute.

This is my first contribution to an open source project. If there is something wrong, please point it out, thank you.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=h1) Report
> Merging [#6572](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/cfa26d2b412ac3494eef06506004ca857c115ad9?el=desc) will **increase** coverage by `1.01%`.
> The diff coverage is `13.15%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6572/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6572      +/-   ##
==========================================
+ Coverage   79.24%   80.26%   +1.01%     
==========================================
  Files         156      156              
  Lines       28130    28243     +113     
==========================================
+ Hits        22292    22668     +376     
+ Misses       5838     5575     -263     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.28% <ø> (ø)` | |
| [...rc/transformers/data/datasets/language\_modeling.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFzZXRzL2xhbmd1YWdlX21vZGVsaW5nLnB5) | `56.32% <10.52%> (-35.52%)` | :arrow_down: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `62.80% <13.33%> (-28.11%)` | :arrow_down: |
| [src/transformers/data/datasets/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFzZXRzL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/tokenization\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYWxiZXJ0LnB5) | `28.84% <0.00%> (-58.66%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `71.84% <0.00%> (-23.17%)` | :arrow_down: |
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `79.69% <0.00%> (-0.26%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (ø)` | |
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `96.59% <0.00%> (+0.68%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.29% <0.00%> (+0.97%)` | :arrow_up: |
| ... and [7 more](https://codecov.io/gh/huggingface/transformers/pull/6572/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=footer). Last update [cfa26d2...e924e84](https://codecov.io/gh/huggingface/transformers/pull/6572?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Thanks for reply and carefully review!
I understand what you and @JetRunner mean now. 
The purpose of submitting this script is to provide a tool that can test `DataCollator` and `TextDataset` (I don't know if I am doing this right). 
I agree with you and think it is not elegant to add a `run_bert_nsp.py` script alone. I will remove this script and support NSP together with MLM objective later.<|||||>Mmmm, your rebase makes the PR touch way too many files now and quite unreadable. You should close and reopen it (once ready) so we can see the diff a little bit better.<|||||>ok i will reopen the pr. thanks~<|||||>I’m very sorry, I didn’t know that pr should be closed before rebase, which resulted in PR touch way too many files.
I didn't notice you have already implement the `BertForPreTraining` in the modeling_bert file before, so I modified `BertForNextSencencePrediction` to support both mlm and nsp objectives. 
These changes have now been revert.<|||||>@sgugger  `DataCollatorForNextSencencePrediction` and `TextDatasetForNextSencencePrediction` now support mlm and nsp together. <|||||>You need to reopen a new PR, this one will forever have all those files diff with all the old commits you rebased, so we can't review it.<|||||>got it.