Hi everyone,
I’m trying to use a small model (nn.Module) to calculate a matrix for an affine transformation between two sets of points. That is, I want to find a valid affine transformation matrix (that can be turned into basic transformation components such as translation, rotation, scale; though only translation and rotation being used for the time being) that turns one set of points into the other.
The points are sort of noisy. That’s why I can’t just use simple linear algebra. I have to use some least squares minimization.
Now, I have successfully built a model that has a 4x4 tensor as nn.Parameter in it that I can optimize using some gradient descent optimizer in order to come up with a matrix that turns one set of points into the other. But the resulting matrix is not a valid spatial transformation matrix (its rotation component does not fulfill certain aspects of a valid rotation matrix) and therefore cannot be dismantled into the particular transformation components (no quaternion_from_matrix possible).
I’ve also tried building a model that has only the transformation components as nn.Parameters, like translation=nn.Parameter(torch.zeros(3)) and quaternion=nn.Parameter(torch.tensor([1.,0.,0.,0.])) and turn them into a transformation matrix according to the well known math behind it. But somehow that way the parameters do not update during a training cycle.
Here’s part of the code:
def transform_from_pq(p, q):
        w, x, y, z = q
        
        x2 = 2.0 * x * x
        y2 = 2.0 * y * y
        z2 = 2.0 * z * z
        xy = 2.0 * x * y
        xz = 2.0 * x * z
        yz = 2.0 * y * z
        xw = 2.0 * x * w
        yw = 2.0 * y * w
        zw = 2.0 * z * w
        
        R = torch.tensor([
            [1. - y2 - z2, xy - zw, xz + yw],
            [xy + zw, 1. - x2 - z2, yz - xw],
            [xz - yw, yz + xw, 1. - x2 - y2]
        ])
        
        T = torch.eye(4, dtype=torch.float32)
        T[:3,:3] = R
        T[:3,-1] = torch.tensor(p, dtype=torch.float32)
        
        return T

and
class TransmatModel(nn.Module):
    def __init__(self):
        super(TransmatModel, self).__init__()
        self.pos = nn.Parameter(torch.tensor([0.,0.,0.], dtype=torch.float32))
        self.rot = nn.Parameter(torch.tensor([1.,0.,0.,0.], dtype=torch.float32))
        
        self.T = transform_from_pq(self.pos, self.rot)
        assert self.T.requires_grad
        
    def forward(self, data):
        if data.shape[1] == 3:
            for_bias = torch.ones((data.shape[0], 1), dtype=torch.float32)
            data = torch.cat((data, for_bias), dim=1)
        x = torch.matmul(self.T, data.t()).t()
        return x[:,:3]

I don’t see where the problem is with that code. I bet it’s got something to do with connections in the graph and that the gradients cannot be backpropagated correctly…
edit Oh, in this case even the assert fails because self.T does not requiregradient_.
Cheers!
Update: Nevermind!
It’s basically the same problem I had a couple months ago. I just couldn’t remember that I had asked this already. Sorry! Solution is here: Trying to find an angle (use of trigonometric functions); optimizer not updating 209