A couple of things you could try:

Apply ReLU before batch normalization, not after as you do now.
Have a separate nn.ReLU() instance for each application of ReLU. Your current code shares the same instance across invocations. Ideally this should not matter, but perhaps there is some sharing of state across these calls? Since you are stuck, you might as well try this and see if it helps.

The first point above may actually make a significant difference. The second point is just something worth trying because it takes only a couple of extra lines of code; it may really not help much.
Edited to add: @arya47 : Do give an update here if either of these helped or not.