Hi, so if I want to learn a matrix M, that will transform an image I into I’ that is ideally very close to the ground_truth image I_g, the forward pass is the following:
M = [1scale 0 dx
0 1scale dy]
The only variables in M are scale, dx, dy
I*M    = I’
loss = I_g - I’
note that this * is actually using grid_sample, and grid_affine but it doesn’t matter here
when I back prop the loss though the gradient will first go to M, and then to dx,dy, scale.
I don’t want it to consider the entire matrix of M as a parameter to update, because for example the first row second column’s value is a fixed 0. For the moment though, its considering 0 to be updatable and I don’t want that. However, if I do M.detach() then dx,dy, and scale won’t be updated/learned…
Anyone know how to do this? Thanks!