Thanks for your speedy reply!
I understand that things like the forward method of a linear layer are using a fast backend. I’m wondering about the slow python code (like Python for-loops and other things that don’t have explicit pytorch functions).  I’m wondering if such code is made faster through torch.jit.script.
Perhaps my point can be seen through the code below. If I had implemented the code below in C or C++, it would be much faster than it is here. Is torchscript not able to turn these slow python for-loops into fast C for-loops? If I ran this saved module after loading it into libtorch in C++, would it run significantly faster? If so, why can’t isn’t python taking advantage of these benefits? I may be missing the point entirely, but do you have any way of making code like the following fast using pytorch?
import torch
from torch import nn
from timeit import timeit

class TestModule(nn.Module):

    def __init__(self, m, n):
        super().__init__()
        self.m = m
        self.n = n
    

    def forward(self):
        val = 0
        for i in range(self.m):
            for j in range(self.n):
                val += i*j
        return val


def purePython(m,n):
    val = 0
    for i in range(m):
        for j in range(n):
            val += i*j
    return val

m = 400
n = 400

# regular pytorch module
model = TestModule(m,n)

# jit'ed module
jitModel = torch.jit.script(model)

# saved jit'ed module
torch.jit.save(jitModel, "saved.pt")
saved = torch.jit.load("saved.pt")


numTrials = 800
print(f"pure python function time: {timeit('purePython(m,n)',  number=numTrials, globals=globals()  )}")
print(f"pytorch model time:        {timeit('model()',          number=numTrials, globals=globals()  )}")
print(f"pytorch jit model time:    {timeit('jitModel()',       number=numTrials, globals=globals()  )}")
print(f"pytorch saved model time:  {timeit('saved()',          number=numTrials, globals=globals()  )}")


On my computer this was the output:
pure python function time: 7.517180057002406
pytorch model time:        7.449000566004543
pytorch jit model time:    13.549996405999991
pytorch saved model time:  14.33033561299817

I wrote the same code in C++ and it ran in less than 1 second.
I also tried importing the pytorch model in libtorch C++ and it took 13 seconds just like in python.
Is there any way to make for loops like this faster in pytorch?
Thanks,
Jack