Adaptive pooling layers take a specific output shape and make sure to pool the input to this shape.
This is useful, if your input tensors have a variable spatial shape.
Average pooling layers work similar to max pooling, but calculate the average of the window instead of the max value.
If it’s not helping, you could of course remove it. Since the output shape is defined in this layer, you might need to use F.interpolate or another method to create the same output shape.