You should be able to identify the colorized pixels by identifying their respective values.
Say the dark red corresponds to a value of 1.0.
Then you can get the indices of red pixels this way:
indices = torch.nonzero(image==1.0)