Using argmax isn’t a good way to select the continuation word. Instead, treat the output of the model as a probability distribution and sample from that. Use np.random.choice with weighted random probabilities.