Since I’ve returned working on this project, I was able to find the memory leak issue. The problem was in my validation loop:
with torch.no_grad():
    for batch in tqdm(testloader, **params):
        x = batch[:, 1:].to(device)
        y = batch[:, 0].unsqueeze(0).T
        y_test_pred = torch.sigmoid(sxnet(x))
        y_pred_tag = torch.round(y_test_pred)
        y_pred_list.append(y_pred_tag.cpu().numpy())
        y_list.append(y.numpy())

Saving the results like this used way more memory than I originally thought. Since this is a binary classification task, my results are 1xn vectors, so I was able to use .flatten() on the numpy arrays before appending, which has solved the excessive RAM use issue.
Unfortunately I’m still facing the problem that if I use num_workers > 1 for my dataloader, it just hangs on doing the second batch. CPU usage is still there but it never finishes the second loop iteration. I have a second dataset that I created by removing the class imbalance from my original dataset, and on that data it works like a charm. From the dataloader’s perspective, the only difference between the datasets should be the number of rows.