After performing Global Average Pooling, I have (N, C=3, 1, 1) (reshaped to (N, C=3, 1)) dimensional features, which I would like to pass to a linear layer. My desired output is of size (N, C, 1) or (N, C). However, I’m unsure of what dimensions to use for out_features.
Dimensions-wise, nn.Linear(1, 1) returns the correct dims, but I’m not sure it makes logical sense. If I’m not mistaken, features of different classes will be sharing the same weight, and this will be treated as a binary problem instead of a 3-class one. How do I perform nn.Linear preserving both the correct output dimensions and the multi-class nature of the problem?
Thanks!