I don’t know, if and where replace_logitsWithLinear is used, but I would recommend to check the input shape of the activation before passing it to the linear layer.
It would probably be the output of self.avg_pool(x).
Also, is your batch size 4096?