Thanks for you’re reply. I have a data set containing 20 classes. Each class is represented more or less equal. So I think this would nor work out.
I already figured out that the network is not working on one of the two data sets. I thought about finding a submaxima of the loss function and tried with really low learning rates. However, the behaviour on that data set remaind the same with much slower cenvergence of the loss, consequently. Thus, I guess, the data set is not working with this network at all. Maybe it is to complex?