In an nn.module, with:
        self.fc1a = nn.Linear(512+2*num_attrs, 512)
        self.fc1b = nn.Linear(512, 512)
        self.fc1c = nn.Linear(2*num_attrs, 512, bias=False)

I would expect both
        xa = F.relu(self.fc1a(torch.cat((x, 0.0*xb), 1)))

and
        xa = F.relu(self.fc1b(x)+self.fc1c(0.0*xb))

to give the same results as
        xa = F.relu(self.fc1b(x))

but only the latter gives the same result while the former consistently leads to a performance degradation. The two statements involves (currently) unused parameters in the same way. Is there something wrong with the first statement? (x and xb are respectively of size 512 and 2*num_attrs in the non-batch dimension). Pytorch version is 1.5.0 and the optimizer is adam.