WHOA!! That really did work…
I have my parameters down from ~11k to ~2k and I can manually control them as well. That was a big help @solvingPuzzles… 
But, the Convolution layer had to be given the number of input channels after AvgPool which happened to be 16 not 16*19=304 (or perhaps did I misunderstood something here?) but it is working now. Just for reference, here are the changes I made:
        ...

        self.bn3 = nn.BatchNorm2d(num_features=16)
        self.relu3 = nn.ReLU()
        self.pool3 = nn.AvgPool2d(2)
        
        self.conv = nn.Conv2d(16, x, kernel_size=1, groups=16)
        
        self.relu4 = nn.ReLU()
        self.dense2 = nn.Linear(x*19,3)

        ...

I’ll just have to evaluate and compare their performance soon enough…