Hello @Gautam_Bhattacharya
that seems like a great project!


Q.  the eps parameter is to avoid division by zero and I don’t need to use expand_as with it?


Yes, that usually is just the regularisation. I’d even leave it as a python float.


Q. The forward pass seems to be working, I was wondering if there are any obvious errors? Do I need to use register buffers for the parameters ?


I think something is up with the indentation, but that is likely only the quoting, I have not checked in great detail.


Q. In the paper they say to insure parameter positivity, they do gradient updates on the log values of the parameters and then take exponentials. How can I go about doing this?


You could use self.log_alpha, log_delta, log_r as the parameter (but ideally init to something close to in 0 instead of 1, too) and then do alpha = self.log_alpha.exp().expand_as(x).
I hope this helps.
Best regards
Thomas