Hi, I’m training a model with BCEWithLogitsLoss. The model has a head like this:
self.head = nn.Linear(config.hidden_size, num_labels)

My loss during this training is about 0.3, when I use the head below:
self.head = nn.Sequential(  #
            nn.Linear(config.hidden_size, 150),
            nn.BatchNorm1d(150),
            nn.Linear(150, num_labels),
        )

The loss goes to 0.6. Is this a normal situation?