Huh, does the second tensor contain values (as opposed to int indexes)? If so, the first tensor is not used, and second.expand(3,-1,-1) (+ .contiguous() if needed) would broadcast across channels. If I misunderstood, you likely need where() or scatter() ops to merge values.