Hello,
ESPNet is not saved by PyTorch script (i mean by torch.jit.save) it has been saved with torch.save already so you can directly use it i think.
https://github.com/sacmehta/ESPNet/tree/master/pretrained/encoder 6
Anyway, I managed to load the encoder this way by previously downloading the pre-trained model from Github.
Regards,
import torch
model = ESPNet(20,encoderFile="espnet_p_2_q_8.pth", p=2, q=8)
example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("model.pt") #Will do the same save a load_dict
#Exemple de save du modele ESPNet avec TorchScript
torch.jit.save(traced_script_module, "scriptmodel.pth")
#Load example with ToarchScript
modelLoaded = torch.jit.load("scriptmodel.pth")

The output is
Encoder loaded!

For ESPNet to be noted that it uses the GPU so tu use CPU i copy Model.py from https://github.com/sacmehta/ESPNet/tree/master/train 2 in my Notebook then i change this instruction
self.encoder.load_state_dict(torch.load(encoderFile)

With
self.encoder.load_state_dict(torch.load(encoderFile,map_location='cpu'))