Thanks for your answer @ptab
So, regarding the point 3: imagine that I’m no longer rolling a dice, but I’m making statements basing on some input properties, something like:
while x.norm(2) < 10:
      x = relu(self.middle_linear1(x))

I will train my module weights following this update(forward) rule. But what if I present to the model a sample that does not satisfy the while condition? I mean, the forward function is something that is only used in training and could be totally different in an eventual future test phase where I’m using the saved model?