Solved by nielsr in post #2 
              
              
                You might take a look at my demo notebook here, which illustrates how to use GPT-J for inference, including adding examples to the prompt.