Solved by my3bikaht in post #4 
              
              
                I think there’s something wrong with labels during training loop because BCELoss = -ln(1-x) if label==0 and -ln(x) if label==1. It can decrease only if your predictions are getting closer to labels you provided. 
If bce loss is dropping to .1 it means average distance between labels and predictions …