Ho sorry I misread the parenthesis, yes this is right.
Yes
I am not sure to understand the issue here, is it that you don’t know x.shape[1] during the init function?
The simplest is for you to precompute it and do define the weights in the init (this will make sure the optimizer will see them properly and stuff).
Otherwise you can do something like this in the forward:

if not hasattr(self, "weight"):
  # First forward
  self.weight = nn.Parameter(torch.randn(self.feature_dimension, x.shape[1]))
x = x = F.linear(x, self.weight)

But the problem is that you will need to wait after the first forward to create the optimizer. And you might need to move self.weight to a different device,