Solved by nielsr in post #2 
              
              
                Hi, 


forward() can be used both for training and inference.

generate() can only be used at inference time, and uses forward() behind the scenes. It is used for several decoding strategies such as beam search, top k sampling, and so on (a detailed blog post can be found here).