I tried this method also
class augmentedX(nn.Module):
    def __init__(self,ni):
        super().__init__()
        self.conv1 = conv_layer(ni,ni//2,kernel_size=1)
        self.conv2 = conv_layer(ni//2,ni,kernel_size=3)
        self.classifier = nn.Linear(ni*7*7,1360)

    def forward(self,x):
        x = self.conv2(self.conv1(x))
        x = x.view(x.size(0),-1)
        x = F.avg_pool2d(x,x.size()[2:]) 
        return self.classifier(x)

In forward pass:
setattr(self,"fc0",augmented1(8*4*2048))

It goes out of memory. I have 8 GB RAM on my GPU server.