Solved by albanD in post #8 
              
              
                I’m sorry I can’t really follow this. 
What would be the pair (x, expected_y) that you want? 
What would be the initial weights w0? 

It means we should never take ReLU output as our final output ? 

It is not advised in general no. Mainly because it does not have any benefit (if you’re value is po…