Hi @Stefano_Savian,
My understanding is that they are NOT always equivalent, but close. For example, if there are batchnorm layers in the network, the batchnorm parameters will be updated differently due to different batch sizes.
There might be other (behind the scenes) differences too.
So, if you have memory, it is probably better to do a single forward pass.