One way to debug it is to try to overfit your network by training with only 1 mini batch repeatedly. If your network is designed properly, your network should converge very quickly and get a loss of nearly 0. Otherwise, there might be serious design issues in your network.