Yes I’m just computing gradients. There is no updating. I don’t have BatchNorm layers. The pipeline is pretty straightforward. Since the model is trained on minibatch of 25, I first copy the new input 25 times and create a batch matrix, then in the output, I only extract the first data in the batch.