In the current pytorch example, all the parameters have to be pre-defined in the class init function or use existing nn.Module, like nn.Linear.  However, this requires us to compute the parameters size correctly for the whole graph, which can be quite tedious and error prone when the model become complex.
Declare the parameters in the forward function seems to be a solution because the intermediate results are already known at that point, but the thing is this might make the parameters be declared every time we run the forward function.
Tensorflow solve this problem by proving a tf.get_variable function.
can we also add something like Module.get_parameter, which create a new parameter or return an existing one if it already exists in the Module.parameters?
def conv_relu(input):
    # Create variable named "weights", its shape can be infered from input.
    kernel_shape = (64, input.size()[0],3,3)
    weights = nn.Module.get_parameter("weights", kernel_shape,
        initializer='uniform')
    conv = F.conv2d(input, weights)
    return F.relu(conv)

https://www.tensorflow.org/versions/r1.0/how_tos/variable_scope/ 25