Solved by googlebot in post #2 
              
              
                img_unfolded.shape 
Out[13]: torch.Size([1, 9, 784]) 
img_unfolded.T.shape 
Out[14]: torch.Size([784, 9, 1]) 

so, .T does the wrong permutation 
gradient of matmul(kernel, img_unfolded) wrt kernel is grads_output.mm(img_unfolded.t()), 3d version of which should use shapes (1,1,784) @ (1, 784, 9) 
I…