Hi @kmfoda. I was not able to investigate into this further. Since the original posting of this, I do know the methods for generating text were overhauled pretty significantly. I haven’t checked but if there isn’t an input parameter to enable generating text on multiple gpu, then I’m not quite sure how to make it happen. Sorry I don’t have better news! If I discover something, I’ll post it here.