Hi again @ptrblck.
So I went with the option: run a dummy forward pass, print the activation shapes, and set the expected values.
def _infer_flat_size(self, model):
    model = nn.Sequential(*model)
    model_output = model(torch.ones(1, *self.input_size))
    return int(np.prod(model_output.size()[1:])) 

However, then I tried an experiment with a setup (fixed seeds) where I know I can reproduce the exact same result. Here I did two different experiments:

One where I initialized the in_features with a specific value (e.g. 301056).
One where I used the _infer_flat_size function() to calculate the value to automatically set it.

But then I get two different training_loss, training_accuracy etc. How come? Is it because of the following line?
    model_output = model(torch.ones(1, *self.input_size))