You are right, unfortunately it is not that easy. The main problem is that jit::scriptmodule and nn::module do not have the same set of methods. I do not know the solution, but in this post they write that eval is not yet implemented in scriptmodule, but they suggest a workaround and there seems to be a brand new fix.
https://github.com/pytorch/pytorch/issues/13991 92