Hi Yun,
Yeah you just got what I want to do. I have updated the code in the question.
It seems impossible to put layers with multiple inputs, just like my SuperPixelPooling into my model with nn.Module, right?
I have to implement the Function version and plug it into my model with super_pixel_pooling = SuperPixelPoolingFunction(input, mask), right?