Obviously, since there is no need for gradient calculation and backward, the built-in modules usually perform faster in the evaluation phase than in the training phase.
However, using  ‘with torch.no_grad()’ and ‘self.eval()’ simply can’t speed up my custom function module(11it/s).
I happened to find that if I set the model to training mode(self.train()) first and perform backward once, the model evaluation speed will double(24it/s). Is there a standard setting method?