I would suggest to ask for help to optimize the nested loop rather than doing it. Realize you can write your custom c++ module (properly done) if you can’t really optimize that nested loop. You can “run” that but consider pytorch won’t be aware of all those operations, thus, will pass wrong gradients…
Lastly just some random links

  
    
    
    Accelerating layers with Numba 
  
  
    Hi all, 
Has anyone had success integrating functions decorated with numba.cuda.jit into a PyTorch workflow? http://pytorch.org/about/ mentions using your ‘favorite libraries…such as Numba’ but I’m not clear on how to actually do that. 
Of course it’s no problem on the CPU to decorate a layer with numba.jit and convert inputs and outputs to and from numpy arrays, although even that doesn’t seem particularly elegant because of all the  numpy/Tensor conversions required (I’m aware they are zero-ov…
  


And I know there is a library which creates cuda kernels to perform optimized operations for a given code, speeding it up, but i forgot the name