Solved by KFrank in post #2 
              
              
                Hi Suresh! 

This is a perfectly reasonable approach and will work fine.  It may well 
introduce de minimis inefficiency, but any such cost will be negligible 
compared with the cost of backpropagation. 

This is also fine.  I would probably use this second approach – not for 
efficiency reasons, bu…