Solved by Zhihan_Yang in post #6 
              
              
                More than a year later, I decided to give this another go. I found that changing ReLU to Sigmoid / Softplus solved the issue. I was able to get the following plot: 
[Screen Shot 2021-12-29 at 4.11.35 PM]