Hi there,
i would like to ask how to customize dataloader in tensorflow.
Now the image paths and labels are saved in csv-file as dataframe.
My first step is get the huge dataframe from many csv files and the images as data will be appended in the dataframe.
Then using a function i get these images and labels from the big dataframes.The images and labels are numpy arrays.
At last i feed the data direktly in model.fit(traindata, trainlbl). The problem comes out when the dataset too big becomes, the cpu will be overloaded and the SIGKILL9 will appear.
I have also tried to put data and label numpy array in tensor via tf.data.Dataset.from_tensor_slices((data, lbl)). And it runs also on cpu, when the data is too big, then also SIGKILL9.
So,  could some one help or give me a hint about how to customize dataset in tensorflow?
best regards,
Chao