When you actually execute the model, the computation graph gets built dynamically (recording all the operations).
What (I think…) takes most of the memory is the outputs of some operations (the edges of the graph). Those will be needed by autograd to compute the gradients so are kept in memory until you call outputs.backward().
Please find more details here 1 and autograd concept better explained than by me.