Ignore this message. The problem was that I had too shallow of a network! Sorry about the noise! It converged after I added a couple of conv layers.

I am having a very similar issue. Initially, I had a 3 conv+relu+maxpool followed by 2 linears + relu and sigmoid for binary classification. It would not learn – all the parameters would be set to something very close to 0, and the output would be “random”. I reduced the size of the network to Conv/relu + Linear/relu + Linear/sigmoid => same problem.