Your use case sound similar to torchvision.transforms.RandomErasing 89, which is used to randomly selects rectangle regions in an image and erases its pixels, so maybe you could reuse their approach for your temporal data.