So, I tried print the pred before and after the _, pred = torch.max(pred, 1)
before it, the pred seems to always have negative values at index 0 and positive values at index 1 per batch and it’s not a deterministic behavior coz it happens vise versa.
These values repeat themselves a lot and they barely change
after it, the pred is only one class per batch (either 0s or 1s) as opposed to the target which is an almost even mixture of both classes, that’s why it still gives an accuracy within the range of 49% - 51%
The loss still reduces by the way, it’s just the accuracy that’s stuck
I’ve also tried a different data normalization method that removed a good degree of skewedness from the extracted features, and I also tried to manually initialize weights across the linear layers, conv2d layers, and both with:
def init_xavier_weights(m):
    if isinstance(m, nn.Linear) or isinstance(m, nn.Conv2d):
        torch.nn.init.xavier_uniform_(m.weight)
        m.bias.data.fill_(0.01)

and then applying it like so: NNModel.apply(init_xavier_weights)
Still no improvement
Edit:
I tried reducing the batch size from 600 to 50 and something strange and interesting happened.
The mean_batch_loss increased to hundreds.
I’ve checked to see if I’m computing the mean_batch _loss correctly and it seems correct to me.