Swarchal:

think you’ve identified the problem with X = X.cuda() , it’s sending all your data to the GPU.


Maybe this code snippet will help u with loading the dataset and then training with only a batch of data:
load data
train_set = MovingMNIST(root=’./data/mnist’, train=True, download=True)
train_loader = DataLoader(
dataset=train_set,
batch_size=config.batch_size,
shuffle=True)
model = …
optimizer = …
loss = …
for epoch, (batch_inputs, batch_targets) in enumerate(train_loader):
batch_inputs = batch_inputs.to(device).float()
batch_targets = batch_targets.to(device).float()