Thank you very much for your help @ptrblck, really appreciate it.
All the inputs should be non-negative - added a check on line 7 to make sure that is the case:
image1300×1286 81.5 KB
Even if I remove the sigmoid nonlinearity and initialize alpha to a value > 1, the issue persists:
image1298×1280 81 KB