How is the interactive GPT-2 implemented?
## ❓ Questions & Help
I came across this online demo from HuggingFace for GPT-2 writing: https://transformer.huggingface.co/doc/gpt2-large. The demo is really amazing, both accurate and fast. My major observation is that the service actually uses user's earlier writing examples in later prediction, almost instantly. I'm very curious how it is implemented? It seems to me that it is not fine-tuned in real-time, then is there some other mechanism behind it? Any ideas are appreciated.
Context examples I typed in:

> Set the a c to low level = the room is very cold
> turn down the volume = the music is too loud

Then when i try:
> turn on the lights = 

It gives me
> the room is too bright

Also, I tried fine-tuning the entire model with much more than 2 examples (around 30), however the result for "turn on the lights = "  after fine-tuning is a lot worse than the demo:

> about 0.016 (lit a second and a 100 pixels)

Is it that the demo only fine-tune, e.g. the very last layer of the model?

Hi, the models are not fine-tuned on the fly. Language models like GPT-2 are very context-aware and are strong at generating words related to the inputs they were given. 

We are not training the models in that demo, we are only using them for inference.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
