Leigh_Shepherd:

        optimizer.zero_grad()



few things can you place        optimizer.zero_grad() just after the for loop for b, (X_train, y_train) in enumerate(train_loader): and secondly your training learning rate is pretty low. try increasing the learning rate to see if anything changes