Hi Ptrblck,
I have tried to train the model on a smaller network dataset (5 classes) but I still get a testing accuracy of 0% (rounded to 0 but there are a few correct predictions) and training accuracy of 51%.  In the first iterations both the training loss and testing loss decrease, and after 20 epochs the testing loss starts to increase again leading to a testing accuracy of 0%. When I inspect the predictions, I found that predictions.unique() often returns 2 or 3 labels, instead of 5.
For instance labels = [000111222333444] and predictions = [444444444000000] where predictions.unique() = [4,0].
Do you have any other idea of what could be wrong?
Thanks again for your help.