After experimenting the mini-batch training of ANNs (the only way to feed an NN in Pytorch) and more especially for the RNNs with the SGD’s optimisation, it turns out that the “state” of the network (hidden state for the RNNs and more generally the output of the network for the ANNs) has one component or one state for each mini-batch element. Thereupon, that is not very clear how Pytorch trains a neural network with a mini-batch and all the more so when the criterion is the SGD (entailing that each iteration a batch element is randomly picked).
Thank you in advance for your answers.