I also tried rmsprop with batch normalization, the same things were observed.
The below is a modified rmsprop employed in Torch7.
square_avg:mul(alpha)
square_avg:addcmul(1.0-alpha, grad, grad)
avg:sqrt(square_avg+eps)
params:mul(1-lr*weight_decay):addcdiv(-lr, grad, avg)

And, the below is an equivalent rmsprop in pytorch
square_avg.mul_(alpha).addcmul_(1 - alpha, grad, grad)
avg = square_avg.add(group['eps']).sqrt()
p.data.mul_(1-group['lr']*weight_decay).addcdiv_(-group['lr'], grad, avg)

Are they equivalent ?
Also, my pytorch code didn’t work at all before 0.19 version due to some run time errors. Is that related to this convergence difference ?