The corresponding PR 1 was merged on Aug 26th, so you could compare the performance using the nightly binary from the day before and after the merge. If you are still seeing the issue, this could narrow it down to this PR.
It’s still strange that you needed to scale down the model layers (I assume you needed to reduce the memory usage), as the PR should save memory and naively I would assume the inplace dropout would save the same amount of memory (haven’t looked into the code deeply yet).