Why do CPUs have multiple cache levels?
I was "enlightened" many years ago when I asked a colleague (a great electronic engineer) why we didn't do fast task switching by having two sets of registers.  His reply was that this would require every regular access to a register to go through an extra gate (to decide which bank of registers you want to hit), making every access slightly slower.Larger registers/caches/memories are slower because they need more address decoding, that time scaling approximately linearly as the storage doubles in size.