Hi @the_coder,
In the training loop, you might also want to zero out the optimizer gradients instead of the network .
So, try:
for epoch in range(10):
    for data in train_dl:  
        X, y = data  
        optimizer.zero_grad()  
        output = network1(X.view(-1,4096))  
        loss = F.cross_entropy(output, y)  
        loss.backward()  
        optimizer.step() 
print(loss)