@tom Hi Thomas,
I hope you still remember this post (not to mention see this one  )
I have been experimenting with this model, and so far it does ok, but still degrades on my baseline. I still have a few optimization tricks to try.
You said:
This way, the backprop will just compute correct adjustments to the log parameters. My understanding is that they did it similarly.
Does this mean that when I do my loss.backward() an inplace log will be taken for the associated parameters, before computing their gradients?
I am just trying to check any possible loose end, though since I do get sensible results, I do think its more of an optimization issue.
Thanks,
Gautam