Indeed you are correct this is the function I had in mind.

Perfect!

Is the aggressive freeing of buffers a performance tweek?

It is a very important memory improvement yes.
In a neural network, most of the memory usage comes from the buffer. So freeing the buffers as you do the backward allows you to significantly reduce the peak memory usage.