This is the list of all Non-Linear activation functions available in pytorch:
https://pytorch.org/docs/stable/nn.functional.html#non-linear-activation-functions 1.
I am sure you know this but just reiterating that we commonly use sigmoid(binary classification) and softmax/logsoftmax (multi-class).
Please let me know if you want more information.