Thank you for your reply!
It seems, while batch_x in shape torch.Size([batch_size, 500, 60, 2130]), in which batch_size is appointed, torch.utils.data.DataLoader functions just like what my DataUtil does.
The question is can pytorch send single mini-batch (shape torch.Size([1, 500, 60, 2130])) once a time to GPU model, to save GPU memory usage. I saw dramatic increase in GPU memory usage while using pytorch than in the Keras case, when the training datas and models are comparative.