This may not have been available at the time of the original discussion, however PyTorch 1.4 has nn.AdaptiveMaxPool2d 133 which is designed to handle the exact use case of variable --> fixed size feature map conversions. You can see my implementation of the entire SPP layer on Github 174