When defining forward my lizard brain always reels at code like:
def forward(self, x):
    x = nn.foo(x)
    x = nn.bar(x)
    x = nn.baz(x)
    return x

or
def forward(self, x):
    return nn.baz(nn.bar(nn.foo(x)) #imagine depper nesting and longer names

and wants to write something like:
from toolz.functoolz import pipe

def forward(self, x):
    return pipe(x, nn.foo, nn.bar, nn.baz) 

or maybe (to structure more complicated models like ResNet or DenseNet):
from toolz.functoolz import compose

class Model(nn.Module):

    def __init__(self)__:
      self.block1 = compose(baz, bar, foo)
      ... # more stuff here

    def forward(self, x):
      x = self.block1(x)
      ... # more stuff here

Now I wonder (since PyTorch uses a dynamic computation graph) whether additional python code (like lambdas and other pure python code as in modules like toolz and functools etc.) incurs some (non-negligible) overhead? This maybe touches upon how the dynamic computation graph is actually build from the python code in forward and I’d really be interested to learn about that.
…all the while my lizard brain hopes it can indeed write more functional code without incurring a performance penalty.