Solved by nielsr in post #2 
              
              
                This is very well explained in this Stackoverflow answer. 
You can also check out our blog post on generating text with Transformers, that also includes a description of the temperature.