Thanks for your reply! I added these lines to my code and it seems that works fine. Even though, the problem that I mentioned with my network still happens. My network is not learning (or learning to predict just one of the three classes). Do you have any idea that can make it happen? I checked my training code with a simpler network and it works well. The problem apparently is with this custom convolutional network.