Performance, Pytorch, Deep Learning, Artificial Intelligence, Tensorboard.
in the case of boolean masking, if our mask is extremely sparse and the original tensors extremely large, the savings in computation from applying the mask might outweigh the price of the host-device synchronization. Importantly, the impact of each optimization should be evaluated on a case-by-case