Following is the main part of my code (for simplicity I remove implementation of model and loss function):
dataset = TensorDataset(data_1, data_2)
train_loader = DataLoader(dataset, batch_size=5000, shuffle=True, drop_last=False)

optimizer = torch.optim.Adam(model.parameters(), lr=0.001, weight_decay=0)
epochs = 100
for index_epoch in range(epochs):
    print index_epoch
    for x_1, x_2 in train_loader:
        optimizer.zero_grad()
        z_1 = model(x_1)  # model is the neural network model
        z_2 = model(x_2)
        loss = get_loss(z_1, z_2)
        loss.backward()
        # print loss_list
        optimizer.step()

I time it using python cProfile, and it turns out 70% of time is on dataloader.py, while only <10% on actual backward propagation.  There is no additional processing of my data, I simply load it with TensorDataset and DataLoader.  And the shape of my data is (5000000, 1).