I thought so! Instead, I’ll need to train a model that can accept a (N,1, *, W) tensor, correct? In order to (deterministically) ignore some one-hots in the original (N,1, length, W) tensor, how should I filter so that gradients are still sensible?
For instance, suppose I have a (1, 1, 10, 3) tensor, and I want to ignore the one-hot (0,1,0). How would I turn:
tensor([[[[0., 0., 1.],
          [0., 0., 1.],
          [0., 1., 0.],
          [0., 1., 0.],
          [0., 0., 1.],
          [0., 0., 1.],
          [0., 1., 0.],
          [0., 0., 1.],
          [1., 0., 0.],
          [0., 0., 1.]]]])

into
tensor([[[[0., 0., 1.],
          [0., 0., 1.],
          [0., 0., 1.],
          [0., 0., 1.],
          [0., 0., 1.],
          [1., 0., 0.],
          [0., 0., 1.]]]])

in a gradient-friendly way?