Found this comment in the Threshold function that was added to allow for ReLU double backward :
# TODO: This class should be removed once THNN function support Variable backward
So my understanding is that I need to write something similar for pooling, at least until THNN support is added.