Dear, All! I am new hand for PyTorch 0.4. Recently, i saw a snippet of code.
import torch
import torch.nn as nn
at_features = torch.randn(128, 196, 512)
at_hidden = torch.randn(128, 1, 512)
at_bias = nn.Parameter(torch.zeros(196))
at_full = nn.ReLU()(at_features + at_hidden + at_bias.view(1, -1, 1))
I was confused by the last line of this code.What the meaning of the ‘nn.ReLU()(at_features + at_hidden + at_bias.view(1, -1, 1))’.  I think the nn.ReLU() is class, the threshold should be the parameter of the class. Why is following the nn.ReLU()? When i construct the neural networks, how to choose the ‘nn.ReLU()’ or ‘nn.functional.relu’?