Hmm you could be right that the preprocess method is not working correctly, but even when I remove the preprocessing and set r = 1 (mixing ratio), so that mixing and preprocessing does not occur. When I test images_cifar == images_mix it always return true, but then still my loss remains the same over each epoch. The mixing class now looks like this (with optplus=False):
def mix(image1, image2, label1, label2, optplus, train):
    image1 = image1.numpy()
    image2 = image2.numpy()
    label1 = label1.numpy()
    label2 = label2.numpy()
    # image1 = preprocess(image1, optplus, train)
    # image2 = preprocess(image2, optplus, train)
    # Mix two images
    #r = np.array(random.random())
    r = np.array([1])
    if optplus:
        g1 = np.std(image1)
        g2 = np.std(image2)
        p = 1.0 / (1 + g1 / g2 * (1 - r) / r)
        image = ((image1 * p + image2 * (1 - p)) / np.sqrt(p ** 2 + (1 - p) ** 2)).astype(np.float32)
    else:
        image = (image1 * r + image2 * (1 - r)).astype(np.float32)

        # Mix two labels
    eye = np.eye(nClasses)
    label = (eye[label1] * r + eye[label2] * (1 - r))

    image = torch.from_numpy(image).float().to(device)
    label = torch.from_numpy(label).float().to(device)

    return image, label

However when I do remove the whole mixing function the loss does seem to decrease, while to me it seems that I am still doing the same then (except that I’m skipping every second image)
EDIT: seems like loss.backward() was not in my loop anymore and len(data) should have been len(images_cifar). The learner does seem to learn something now without using the preprocess function. I will be trying to fix that now