Hi All
I am working on hyperspectral image as input of my NN.
this is the shape of my data set (65000,1,92) - 65000 is number of samples (signal) , 1 is channel and 92 is length of my signal.
When I trained my model it turned out that error of test data is less than error of training.
I assume that it has to do with how i shuffle my data. because signals are distributed randomly in my pixels. (every single pixel is a signal)
below you can see my dataloader code, has anyone have any idea if it is OkAY?
data_set = torch.utils.data.TensorDataset(new_input_tensor,new_target_tensor)
n_train = int(len(data_set)*0.80)
n_test = int(len(data_set)) - n_train
train_data_set, test_data_set = torch.utils.data.random_split(data_set, [n_train, n_test])
train_loader  = torch.utils.data.DataLoader(train_data_set, batch_size = 2621, shuffle=True, drop_last=True)
test_loader  = torch.utils.data.DataLoader(test_data_set, batch_size = 2621, shuffle=True, drop_last=True)
Thank you,