Hello all, I want to perform PCA on the feature maps / weights in the hidden layers of GANs with for instance a single component, in order to see if there is possible to optimize the GANs architecture with PCA. Currently working with the ESRGAN on images, and training takes a long time.
I was wondering how to perform PCA on convolution layers in the network.
For instance if this is my model, where can I perform the PCA? Or should I do it in the training for loop?
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = nn.Conv2d(1, 3, 3, 1, 1)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(3, 6, 3, 1, 1)
        self.pool2 = nn.MaxPool2d(2)
        
        self.conv_trans1 = nn.ConvTranspose2d(6, 3, 4, 2, 1)
        self.conv_trans2 = nn.ConvTranspose2d(3, 1, 4, 2, 1)
        
    def forward(self, x):
        x = F.relu(self.pool1(self.conv1(x)))
        x = F.relu(self.pool2(self.conv2(x)))        
        x = F.relu(self.conv_trans1(x))
        x = self.conv_trans2(x)
        return x