Great explanation! Let me rephrase this approximation strategy:

To approximate the first E, we iterate over  all (X, Y) in dataset
To approximate the integral while ignoring the second E, we give each (X, Y) a random delta on the fly. This means each example will be paired with multiple deltas during the whole training process with multiple epochs. Overall, it approximates the integration over the space of upper cased Delta for each (X, Y).

Then, we do it in batches. At each batch, we use the average L_{imp} to back-propagate. I hope I didn’t misunderstand your explanation. Thanks.