Well the second argument is supposed to be the indices of the label. So it cannot really be a floating point type.
The Long type is the only one we have that guarantees that you will be able to index any Tensor.
Note that these labels should be fixed, so you could precompute the poolings for all of them and just load the precomputed version no?