It does smoothen out in the end which is encouraging.
As far as the accuracy is concerned, could you show the way you calculate accuracy in more detail. For example, it is not clear to me as to what ‘l’ is. I am assuming label is the ground truth and ‘l’ is the number of classes. The code, if(prediction[i] > 0.5 and label[i] == 1) is not an ideal way to compute accuracy. A better idea is to take the index of the maximum value of prediction and use that as the predicted label as shown below. This is done to ensure that you can compute the predicted labels independent of the ground truth labels which is not the case in your code.
import numpy as np
import torch
no_of_classes = 2
target = torch.from_numpy(np.random.randint(0,1,64))
predicted = torch.from_numpy(np.random.randn(64,no_of_classes))
### Accuracy is computed after this
no_correct_predictions = torch.sum((predicted.argmax(dim = 1) == target).float())
acc = no_correct_predictions/ predicted.shape[0]