Hello there!
I’m not entirely sure how nn.Module works in detail, so I wanted to ask one question.
Are there any operations that are not allowed or should not be done in forward in order to make backward work properly?
For example (assuming my data is of shape (n, 10), e.g. n observations 10 features each) will backprop work just fine with such definition of forward function?
class SimpleNet(nn.Module):

    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(5, 2)
        self.fc2 = nn.Linear(5, 2)
        self.fc_final = nn.Linear(4, 1)

    def forward(self, x):
        x1_1 = F.relu(self.fc1(x.narrow(1, 0, 5)))
        x1_2 = F.relu(self.fc2(x.narrow(1, 5, 10)))
        return self.fc_final(torch.cat([x1_1, x1_2]).view(-1, 10))

Are there any particular limitations with regard to forward? If there are some, I’d like to ask how to achieve an architecture similar to the one presented in a proper way.