Hello! I have this code:
import numpy as np 
import matplotlib.pyplot as plt
import pandas as pd
import torch 
import torch.nn as nn 
import torch.nn.functional as F 
import torch.optim as optim 

f = pd.read_hdf("data.h5")
dt = f.values
data = torch.tensor(dt[0:700000])
data_train_x = data.t()[0:2100].float().cuda()
data_train_y = data.t()[2100].float().cuda()

So I want to load the data to use it for training of a NN. When I run this part I am getting this error:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-cb8efd990293> in <module>
----> 1 data_train_x = data.t()[0:2100].float().cuda()
      2 data_train_y = data.t()[2100].float().cuda()

RuntimeError: [enforce fail at CPUAllocator.cpp:56] posix_memalign(&data, gAlignment, nbytes) == 0. 12 vs 0

I assume that somehow I am running out of memory. The training data is (for the x variable) 700,000 arrays of length 2100 each. For the y variable 700,000 arrays of length 1 each (1 or 0). Is there a way to get around this and use my whole data? Maybe somehow load the data during the training or something? I assume i can use less than 700,000 but I kinda use all the available data. Any advice? Thank you!