Solved by ptrblck in post #4 
              
              
                Not necessarily and you could check all intermediates to have a look what is “happening” in the model: 
class smallDense(nn.Module):
  def __init__(self, d, d_star):
    super(smallDense, self).__init__()
    self.fc1 = nn.Linear(d,4*d_star)
    self.fc2 = nn.Linear(4*d_star,1)
  
  def forward(sel…