The graph is actually created while forwarding, what the backward is doing is just traversing it.
The difference between retain_variables (or retain_graph which is its new name) being True and False is that in the first case, the graph will be left unchanged during backward, in the second case, the graph will be destroyed as you go through it (to reduce memory usage) and thus you won’t be able to reuse it.