You should be able to multiply the conv output with your custom mask.
If you want to use a different mask for each batch element, your mask should have a batch dimension and different masks for each sample in the batch.
Are you seeing any issues with this approach?