As I understand it, the output of each kernel of a CNN is equivalent to exactly one channel of the CNN layer output. The paper wants to integrate over the kernels of the CNN, and they suggest using dropout as an approximate method. So I suppose you could loop over the channels of the output applying dropout to each channel individually - that would apply dropout to each kernel of the CNN individually, but that approach is mathematically equivalent to applying dropout to the entire layer output in one go. The last but one paragraph of section 5 that explains how to apply dropout to the CNN layer seems to support my interpretation.
This is my summary of the paper as I understand it. I could be wrong, but I am fairly sure I have understood it correctly.
In training apply standard dropout after every CNN and fully connected layer of the network. Train with the usual softmax loss combined with L2 regularisation.
When making predictions at test time, don’t turn off dropout as we usually do, instead make several runs through the network with dropout turned on, thus collecting several different predictions for the same input. The average of the predictions can be used as the predicted value. The spread of the predictions can be used as a measure of uncertainty. This is what they call Monte-Carlo dropout.
The number of runs needed for optimal test performance must be determined by testing. Their tests suggest that 20 runs through the network generally produces a significant decrease in test error, and that more than 100 runs is generally unnecessary.
The prediction runs can be efficiently run in parallel by making a batch containing the same image repeated many times.