Thanks ptrblck! I also found that torch.distributions.Multinomial does what I want more directly:
m = torch.distributions.Multinomial(20, torch.tensor([1/6]*6))
m.sample()

I am still not sure how to work with weights (not probabilities) though. Using the previous example weights = torch.tensor([0, 10, 3, 0], dtype=torch.float) we have w1 = 0, w2 = 10, w3 = 3, w4 = 0. Are the probabilities p1 = 0, p2 = 10/13, p3 = 3/13, p4 = 0 in this case?