Thank you. That solved that issue but now I am running into a new one.
segment = mask*color # should have shape 1, 3, 100, 100
RuntimeError: The size of tensor a (100) must match the size of tensor b (3) at non-singleton dimension 3

Also, the shape of mask after mask = mask.unsqueeze(1) is [1, 3, 100, 100] instead of [1, 1, 100, 100] but I am not sure why.