Thanks again! Of course, you were right with your derivation. I somehow applied scalar derivation rules for vectors which is of course wrong. My bad, sorry.



 albanD:

In your case passing either a tensor of all ones will let you know how sensitive is the sum of the output wrt each input pixel.


In other words: A pixel with high dy/dx contributes a lot to the final network output. Correct? So when visualizing x.grad I will obtain an “attention” map of the input image.
 
Input Image   ------------------------------------------------- dsum(y)/dx (i.e. x.grad after y.backward(torch.ones(…)))
Therefore, looking at these images, I can say that the network pays attention to edges (surprise!). Any idea what that wiggly stuff could relate to? Maybe some texture-filters?