Could you flatten the tensors starting in dim1? This might not change the high loss value (depends on the used criterion), but would most likely fit the expected input shape of [batch_size, *]. I would recommend to check some outputs and targets manually and make sure that the values are in the expected range and that no unwanted permutation was appiled somewhere in the code.


Your model might create large intermediate tensors, which would create this OOM issue. The number of input features is also relatively large in the linear layer, which would also point towards a high memory usage.