Even if you are freezing layers, you still have to compute the full forward pass to get the signal of the non-frozen layers. You save some time since you don’t compute the full backpropagation but nothing else.
What you can do (having everything frozen), is to pre-compute the output of the last frozen layer (since that will be static epoch per epoch), but it will require lot of memory since there are no compression formats such us jpg developed for that.