Thanks for the feedback. I am afraid that I loose valuable information when pooling the 40x40 pixels image in the pooling layer with nn.AdaptiveAvgPool2d. I think it might be more usefull to add additional conv layers. As I am quite new to ML I am looking for the right combination of layers for my specific input.