You have forgotten to zero out the gradients, so that they will be accumulated in the .grad attributes of all parameters.
If you add optimizer.zero_grad() into your training loop, the training starts to converge:
    for images, labels in trainloader:
        optimizer.zero_grad()
        out = model(images)
        loss = criterion(out, labels)
       # Acc training
        _, pred = torch.max(out, dim = 1)
        equals = pred == labels
        train_acc += torch.mean(equals.type(torch.FloatTensor)).item()
    
        # Loss training
        train_loss += criterion(out, labels).item()
        
        # Update weights
        loss.backward()
        optimizer.step()