Can we fine tune GPT2 using multiple inputs?
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hello. I don't think this is possible looking at the code, but to make sure is it possible to use multiple input texts to fine tune GPT-2? For example, I have 5 news articles. Do I submit them all in one document or should I separate it out? I feel like the latter makes more sense especially if the articles are all written differently. 

Thanks!

You can merge them in the same document but add a token that indicates it's the beginning and end of each article. This will allow the generator to understand the concept of a document being a group of text and there are many of those groups in your document.

**Example (assuming one line is one document):**

```
<sod> Today was a good day! <eod>
<sod> Tomorrow is going to be even better <eod>
```

Then at the text generation step (e.g. using ```run_generation.py```), you can specify a ```stop_token``` argument if you want to generate one document at a time.<|||||>Fantastic! Thank you for sharing @enzoampil 👍 