agt:

Where does 3x3 come from?


Sorry, that was incorrect. With input shape (3,1) you have 1 x out_features map. With usual shape (*,3) you have 3 x out_features map.



 agt:

What does it mean to scale gradients? Do you mean in a weighted sum?


If you multiply elementwise losses by some tensor C, all gradients are also multiplied by C. So, that’s one of the ways to change gradients.



 agt:

I don’t understand what is “entangled” and why it affects scaling gradients.


Nevermind. Basically, for 3d -> 1d function, expresed with a neural net, 3 inputs usually only work together.