How about
mask = (A < torch.arange(0, A.size(1), device=A.device, dtype=A.dtype).unsqueeze(0))
masked_A = torch.where(mask, A, torch.zeros(1,1, dtype=A.dtype, device=A.device))

where

the arange will give you a vector of j and unsqueezing makes is broadcasteable to As shape,
the comparison gives a matrix of the same shape as A with the desired mask

torch.where selects the masked inputs and zero otherwise.

The reason to use where instead of mask.float() * A is to also mask away NaNs.
Best regards
Thomas
[Edit: Depending on whether masked means “stays if the condition is set” or “is set to zero”, you might use <= in place of >.]