Good point, thanks. I missed that adding channels decreased the fan-in. However, num_attrs is 312 so math.sqrt((512+2*num_attrs)/512) is just a bit less than 1.5. Trying:
        self.fc1a = nn.Linear(512+2*num_attrs, 512)
        with torch.no_grad():
            self.fc1a.weight *= math.sqrt((512+2*num_attrs)/512)
            self.fc1a.bias *= math.sqrt((512+2*num_attrs)/512)

still did not give the same result as in the two other cases. I eventually figured out where the difference was coming from while trying this though (fc1b was used elsewhere and combined gradients helped so not using it here had a negative impact). Thanks.