You can wrap the forward pass into with torch.no_grad() to  avoid storing the intermediate activations and create the computation graph. However, this shouldn’t change the used operations.
If you are using a GPU, you could use torch.backends.cudnn.benchmark = True to let cudnn benchmark all workloads and select the fastest algorithm. Note that the first iteration will observe an overhead due to the profiling.