There really isn’t enough info in your post to give an extensive answer.  But generally, have you tried removing the sigmoid function and have you made sure that your loss function accepts inputs between 0 and 1?  Many of the loss functions require log probabilities.  Hopefully this helps but if not, you will probably have better luck if you post an example input and your code.