When adding attributes to a nn.Sequential objects (because you want to access them later), attributes that are nn.Module themselves are automatically added to the sequence. This can be convenient, but it’s easy to miss.
Correct::
In [4]: class Net(torch.nn.Sequential):
   ...:     def __init__(self, activation):
   ...:         sequence = collections.OrderedDict(activation=activation)
   ...:         super().__init__(sequence)
   ...:

In [5]: Net(activation=torch.nn.Tanh())(torch.ones(2, 2))
Out[5]:

 0.7616  0.7616
 0.7616  0.7616
[torch.FloatTensor of size 2x2]

Also correct, but undocumented:
In [4]: class Net(torch.nn.Sequential):
   ...:     def __init__(self, activation):
   ...:         super().__init__()
   ...:         self.activation = activation
   ...:

In [5]: Net(activation=torch.nn.Tanh())(torch.ones(2, 2))
Out[5]:

 0.7616  0.7616
 0.7616  0.7616
[torch.FloatTensor of size 2x2]

Incorrect output, but forward will run without errors:
In [17]: class Net(torch.nn.Sequential):
    ...:     def __init__(self, activation):
    ...:         super().__init__(activation)
    ...:         self.activation = activation
    ...:

In [18]: Net(activation=torch.nn.Tanh())(torch.ones(2, 2))
Out[18]:

 0.6420  0.6420
 0.6420  0.6420
[torch.FloatTensor of size 2x2]

This behavior is not found in the documentation 8. Is it intentional? If so, should it be documented?
PyTorch version 0.3.0.post4.