What is freed is everything that is not accessible from python in any other way than during the backward pass.
Some operations store extra temporary buffers, but also intermediary Variables that are not accessible from python will be freed.