Hello  James,
Thanks for replying me, I build the pytorch from source and the version is 1.4.0a0+93db2b8
the model is a modified transformer. I transfer the model to torchscript model by using script_model = torch.jit.script(model). (and definitely some other work for jit-compatible)
by the way, I use the torch.quantization.quantize_dynamic to quantize the model, then the first pass
costs about 12.5s.
this is parts of print(script_model):

image505×787 9.6 KB

I only use torch.jit.script(), should I mix the trace and script?
and how can I submit the serialized model file to you? for some reason, I can’t give you the
modifiled transformer, but I can provide the orginal transformer serialized model file for you which first pass costs 32.83s while the others take about 9s(yeah, it’s about 20s again). The origin and modifiled transformer are same net architecture but have different inference functions, so  they have same problem in first pass.