albanD:

I think you main problem is that Tensorflow input has size [708, 256, 3] with [batch, width, channels] . While pytorch expect [batch, channels, width] .


See here, the thing is in TensorFlow, 708 is not the batch size. I have a 3-D input (3 channels, of 708 rows and 256 columns.
Anyway I have changed as you suggested, but I get this error now
RuntimeError: multi-target not supported at C:\w\1\s\tmp_conda_3.7_104508\conda\conda-bld\pytorch_1572950778684\work\aten\src\THNN/generic/ClassNLLCriterion.c:22
at
—> 25     loss_train = criterion(output_train, ytrain)