@setu4993 Thank you for the insights! Can you please elaborate on how you have managed to make .generate() use Elastic Inference? Are you simply calling this method after loading the model with torchscript=True? We tried that but EI is not used that way, all inference takes place on the CPU.