Michael_Hsi:

I understand that torch.eq() will break the gradient, hence question 1 is ‘how can I circumvent this issue’


Yes, torch.eq will return a BoolTensor and will thus detach the output from the computation graph. Unfortunately, I don’t know how to avoid this besides trying to com up with a custom backward function, i.e. you would have to define gradients for the equality/inequality of the values.



 Michael_Hsi:

and secondly, does torch_tensor.bool().int() break gradient as well?


Yes, as gradients are defined for floating point types.
Generally, you could check if an operation detached the result by printing the .grad_fn of the inputs and outputs of a function. If the .grad_fn shows None, the tensor is detached.