Hey t.g., I reached to a fellow named Elliot Waite 31 on YoutTube and his video and answer to my comment very much clarified things for me.
In summary, think of setting requires_grad = False as a way of telling the L2’s parameters to not care about gradients (since the gradients are 0, it will never update). Now, what I was missing, is that it helps out greatly to think of L2 as a bunch of input parameters (those which you are freezing) connected to a bunch of computation nodes, and some of those computation nodes will have the output of L1 as another input. When you set the requires_grad = False, you are NOT freezing the gradients on the computation blocks, therefore the necessary gradients still get passed backward to L1. Elliot pointed out that you will get an error if you try to set requires_grad on a computation.