<p>I am given a dataset in which each instance consisting of 5 sentences. The goal is to predict the sequence of numbers which represent the order of these sentences.</p>
<blockquote>
<p>For example, given a story:</p>
<p>He went to the store. He found a lamp he liked. He bought the lamp.
Jan decided to get a new lamp. Jan's lamp broke.</p>
<p>your system needs to provide an answer in the following form:</p>
<p>2   3   4   1   0</p>
<p>where the numbers correspond to the zero-based index of each sentence
in the correctly ordered story. So &quot;2&quot; for &quot;He went to the store.&quot;
means that this sentence should come 3rd in the correctly ordered
target story. In This particular example, this order of indices
corresponds to the following target story:</p>
<p>Jan's lamp broke. Jan decided to get a new lamp. He went to the store.
He found a lamp he liked. He bought the lamp.</p>
</blockquote>
<p>My initial idea is to extended the NSP algorithm used to train BERT, to 5 sentences somehow. I can't find an efficient way to go about doing so. All suggestions would be appreciated.</p>
<p>Thank you!</p>

CC BY-SA 4.0
2022-03-10T09:33:36.477Z
Is this a homework problem? If yes, you should tag your post with [tag:homework].
nbro
CC BY-SA 4.0
2022-03-21T00:35:51.283Z
No, its for a personal project. Figured it out though: turns out its just using a custom head on the BERT model
lostpatriot
CC BY-SA 4.0
2022-03-21T10:30:32.653Z
Feel free to write a formal answer below to your own question ;)
nbro
<natural-language-processing><recurrent-neural-networks><bert>
Next Sentence Prediction for 5 sentences using BERT
lostpatriot
# Next Sentence Prediction for 5 sentences using BERT

(Asked by: username_0 on 2022-03-09)

I am given a dataset in which each instance consisting of 5 sentences. The goal is to predict the sequence of numbers which represent the order of these sentences.

> For example, given a story:
>
> He went to the store. He found a lamp he liked. He bought the lamp.
> Jan decided to get a new lamp. Jan's lamp broke.
>
> your system needs to provide an answer in the following form:
>
> 2 3 4 1 0
>
> where the numbers correspond to the zero-based index of each sentence
> in the correctly ordered story. So "2" for "He went to the store."
> means that this sentence should come 3rd in the correctly ordered
> target story. In This particular example, this order of indices
> corresponds to the following target story:
>
> Jan's lamp broke. Jan decided to get a new lamp. He went to the store.
> He found a lamp he liked. He bought the lamp.

My initial idea is to extended the NSP algorithm used to train BERT, to 5 sentences somehow. I can't find an efficient way to go about doing so. All suggestions would be appreciated.

Thank you!

**Tags:** natural-language-processing, recurrent-neural-networks, bert

> **Comments:**
> **username_1**: Is this a homework problem? If yes, you should tag your post with [tag:homework].  
> **username_0**: No, its for a personal project. Figured it out though: turns out its just using a custom head on the BERT model  
> **username_1**: Feel free to write a formal answer below to your own question ;)  

---

## 0 answers

