Is the number of views known before the forward pass or is it changing for each batch?
In the former case you could just pass the number of views to your model’s __init__ method and define the linear layer there as usual.
In the latter case, you could try to use an adaptive pooling layer with a specific output size, so that your linear layer will get the same number of features for differently shaped inputs.
As you said, you could also use the functional API and define your linear weights as nn.Parameters in your model. However, I’m not sure how you would like to change these during training.
In case the next input would be larger, would you like to initialize the new values randomly while the other part of your weight matrix has already been trained?