I think I just got the answer.

import torch.nn.functional as f
a = torch.randn(2,3)
norm_a = f.normalize(a,dim=0,p=2)

where p=2 means the l2-normalization, and dim=0 means normalize tensor a with row.