The first thing you should try is to overfit the network with just a single sample and see if your loss goes to 0. Then gradually increase the sample space (100, 1000, …)
Also instead of printing the losses, try to visualize them on a single graph (both train and val). This will give you better insights.
And try to use learning rate decay after a few epochs. Maybe it could help.