Hi all,
I have a multivariate regression problem in hand where a multivariate function takes in the image,pointcloud as tensors and transformation matrices as tensors and outputs the projection of the pointcloud in the image.
The model parameters are the 3 translational parameters of the transformation matrix.
I have set up a nn.module class wherein the forward undergoes a series of filtering (one such filtering example is shown below), the model has three parameters with requires_grad = True, and I am not detaching or switching over to another library in the middle.
Example of filtering ops i do within the forward pass
    img_2D_points_tensor_with_depth = img_2D_points_tensor_with_depth[img_2D_points_tensor_with_depth[:, 0] < leftImagetensor.shape[1]]


When i debug the forward pass, I can see that my output from the forward pass has the flag requires_grad = True too, but after i call loss.backward(), my model parameters are not updated by the optimiser. (I use SGD)
What do i look for in the forward pass to check if the differentiability is lost somewhere? Does a filtering op like the one i have mentioned above break differentiability anywhere?
My loss function is L1loss
You can find the code below
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        #parameters of the model
        self.t = nn.Parameter(torch.randn((3,1), requires_grad=True, dtype=torch.float))

    def forward(self, Left_imgtensor, Right_imgtensor, veloScan, R_0, t_0, K_00, R_cam1_cam0, t_cam1_cam0, K_01):

        Tr_cam0_velo = torch.cat([R_cam0_velo, self.t], 1)
        homo = torch.tensor([0, 0, 0, 1]).unsqueeze(0).float()
        Tr_cam0_velo = torch.cat([Tr_cam0_velo, homo], 0)

        output = multivariate_function(Left_imgtensor, veloScan, R_0, t_0, K_00, Tr_cam0_velo)

        return output 

torch.manual_seed(42)

model = Model().to(device)
print(model.state_dict())

lr = 1e-5
n_epochs = 5

optimiser = optim.sgd(model.parameters(), lr=lr)

for epoch in range(n_epochs):
    model.train()

    yhat = model(left_imgtensor, Right_imgtensor, VeloScan, R_0, t_0, K_00, R_cam1_cam0, t_cam1_cam0, K_01)
    yhat = yhat.view(1,1,512,1392)
    loss_fn = compute_loss(yhat, y_target)
    print(loss_fn)
    for p in model.parameters():
        if p.grad is not None:
            print(p.grad.data)
    loss_fn.backward()
    optimiser.step()
    optimiser.zero_grad()