Solved by agt in post #2 
              
              
                Make sure both your inputs and your outputs are normalized and try it again. How to normalize a tensor to 0 mean and 1 variance? - #2 by ptrblck