Solved by Julien_Despois in post #3 
              
              
                Just make sure that you’re using a Binary Cross-Entropy loss, as the MSE loss can have some issues with the Sigmoid (source)