Thanks for the hint not using old logic. I should not be too lazy when copy/paste 
Yes I have tried to change the hyper parameters like activations and learning rates and batch sizes. Sadly  this does not improve the model. I have also tried to add additional dense layers without success. If there is no mistake in the way I have formulated loss, I am kind of fine with that. Sometimes you might have to accept that a problem is not solvable.