Things with weights are created and initialized in __init__, while the network’s forward pass (including use of modules with and without weights) is performed in forward. All the parameterless modules used in a functional style (F.) in forward could also be created as their object-style versions (nn.) in __init__ and used in forward the same way the modules with parameters are.