Ok, I understand.
But is there a way to construct this in the constructor and not the forward pass?  Because if there are many input- and output-channels, this might be a lot of work for every forward pass.