DCurro:

During each iteration of the outer loop, trainloader and valloader will continue from where they left off in the previous loop.


I don’t think they will continue, since the loop is re-initialized. If you want to continue from the last break, you would have to use something like a generator. Did you manage to get the desired behavior?
If so, could you share some code? 
Here is a small example showing the last samples are dropped:
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision import transforms

train_dataset = datasets.FakeData(size=10,
                                  image_size=(3, 24, 24),
                                  transform=transforms.ToTensor())
test_dataset = datasets.FakeData(size=10,
                                 image_size=(3, 24, 24),
                                 transform=transforms.ToTensor())

train_loader = DataLoader(train_dataset)
test_loader = DataLoader(test_dataset)

for epoch in range(3):
    for batch_idx, (data, target) in enumerate(train_loader):
        print('epoch {} train batch {}'.format(epoch, batch_idx))
        if batch_idx == 5:
            break
    
    for batch_idx, (data, target) in enumerate(test_loader):
        print('epoch {} test batch {}'.format(epoch, batch_idx))
        if batch_idx == 5:
            break

Comment the condition and break statement and you will see that each loader returns 10 samples.