Hi!
I’m confused to see that two weight checkpoints from the same network can have very different inference times on CPU. I’ve seen cases where one checkpoint makes inference consistently 2X slower than another checkpoint. I have not seen this kind of inconsistency on GPU.
I have prepared a minimal working example for which w2.pth makes inference 1.5X slower than w1.pth.
https://github.com/lemairecarl/paratope-pred/archive/cpu-perf-tests.zip
From my investigation, it seems that checkpoints taken earlier in training are faster to do inference with. Does that make sense? Is there some magic optimization tricks which I’m not aware of?
All the best,
Carl