I looked at the source code 5 and came to the conclusion.
def cross_entropy(input, target, weight=None, size_average=None, ignore_index=-100,
                  reduce=None, reduction='mean'):
    if size_average is not None or reduce is not None:
        reduction = _Reduction.legacy_get_string(size_average, reduce)
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)

From the source code, I’m sure that

Or alternatively the CrossEntropyLoss function in PyTorch will do that (computes the probability of prediction of model)  automatically ?

This is the right answer.