Hi,
I’m not sure to understand exactly what happens here but it feels like you’re measuring more noise than anything:
Try adding outside of your function data = torch.rand(128, 784), torch.rand(128).long() and replace the dataloader step with x, y = data.
Just to reduce memory use on that side, then the backward pass runs 5x faster on my machine.
Now changing your custom backward to do nothing and return None, None, None actually has a similar runtime as the original linear layer’s one (with a lot of noise depending on the runs though).
So I would guess your custom backward looks slower because you do such small ops that calling 4 python ops are actually the most expensive thing that happens here. What do you think?