The Module module of PyTorch is built on top of functional so there is a bit more overhead but it is completely dwarfed by the time spent on computing linear, convolutions, RNNs and other layers. (We’re speaking into seconds vs minutes/hours/days here)