my point isn’t really about coding style per se, but more about correct PyTorch semantics. The motivation for my followup question is more like the o.p. : What goes into init() and what goes into forward()? as I found that confusing as well, especially coming from Keras. I think it boils down to wether a layer (module??) holds state or not as you put it. Maybe there are other reasons I have not learned yet??. For example, you can even add new layers in the forward(), e.g. F.conv2d(…), but that layer weights are lost once the forward() exits, so it can’t be used as part of the trained model for example at prediction time. So the semantics -not the syntax- of what goes into init() vs what [should] go into forward() is the main point of the question.