Hi Asaf!



 abuchnick:

For some reason, we can’t get the optimizer to optimize the noise,
    def generate_attacked_poses(self, traj_dir_path: str, traj_num: str, noise_, attacked_imgs_save_dir, epoch_num: int = -1):
...
            attacked_poses[i+1] = global_pose_attacked[0:3, :].detach()
            # update
            attacked_img1 = attacked_img2
        return attacked_poses
...      
                curr_traj_attacked_poses = self.generate_attacked_poses(curr_traj_dir, str(traj_num), noise_, save_dir, str(epoch_num))
                curr_traj_attacked_poses.requires_grad_(True)



The various calls to .detach() along the code path that generates
attacked_poses “breaks the computation graph” and setting
requires_grad = True thereafter doesn’t repair the damage.  That
is to say, no gradients will be backpropagated through the .detach()
calls, so gradient descent will not be able to optimize noise.
May I suggest that you look at pytorch’s autograd documentation,
in particular the Computational Graph section 1 of pytorch’s autograd
tutorial?
Best.
K. Frank