Generate Variable Length Text With GPT2
This might be obviously explained in the documentation, but I've been browsing through the code for a while and can't seem to find a resolution, so thank you in advance for your help.

As demoed with Write with Transformers, it seems to generate variable length text suggestions. I was wondering how this would be possible with the transformers library given, and how it would be possible to interface the largest version of GPT2 to do so.

Thank you!
Hi! In Write With Transformer, we use the context to predict the following token. We then add that token to the initial context, to generate the following one. This way we can generate long sequences according to a given token.

In that app we stop generating tokens once we have reached a given time, or once we have seen an end of sentence token. We, therefore, don't generate variable length text suggestions, we just adjust the batch according to end tokens identified in our results.<|||||>Thanks for the explanation!