Hi, the $ in the original paper is a symbol representing an additional token that you need to create and add for the finetuning of the model. On the contrary to BERT and more recent models, GPT/GPT2 don’t have a native separation token that you can use to separate pair of sentences.
You have to manually add an additional token to the vocabulary (you can use tokenizer.add_tokens() or tokenizer.add_special_tokens() in transformers for instance) and then train the model on a dataset with pair of sentence to learn an embedding for this new token. You also need to concatenate the sentences yourself with the added token or the pattern you have selected to concatenate pair of sentences.