Upon digging into the details, it appears that the models are trained with CrossEntropyLoss which has SoftMax built in.
Imagenet training script:
Criterion defined at: https://github.com/pytorch/examples/blob/master/imagenet/main.py#L91 189
Loss defined at: https://github.com/pytorch/examples/blob/master/imagenet/main.py#L195 168
CrossEntropyLoss doc: http://pytorch.org/docs/master/nn.html#torch.nn.CrossEntropyLoss 621