You are correct that the dispersive approximation is just a perturbative expansion of the Jaynes-Cummings Hamiltonian in the parameter $g/\Delta$ following unitary transformation. Instead of focusing on the mathematics, I'd like to give you an intuitive feel for what these terms mean. Let us rewrite our effective Hamiltonian as $$ \begin{align}H_{\text{eff}}/\hbar &= \omega_ra^\dagger a+\omega_{a}\frac{\sigma_z}{2}+\frac{2g^2}{\Delta}(a^\dagger a)(\frac{\sigma_z}{2}) \&= (\omega_r+\frac{2g^2}{\Delta}\frac{\sigma_z}{2})a^\dagger +\omega_{a}\frac{\sigma_z}{2} \&=\omega_ra^\dagger a +(\omega_a+\frac{2g^2}{\Delta}a^\dagger a)\frac{\sigma_z}{2}\end{align}$$ where we have neglected the $\frac{1}{2}$ term that you mentioned above (note that the $\frac{1}{2}$ term is actually the ac Stark shift of the atom due to vacuum fluctuations in the resonator). Let us define $$\omega_{r}'=(\omega_r+\frac{2g^2}{\Delta}\frac{\sigma_z}{2})$$ and $$\omega_a'=(\omega_a+\frac{2g^2}{\Delta}a^\dagger a).$$ Notice that our primed 'resonance frequencies' contain operators. Now our Hamiltonian reads $$H_{\text{eff}}/\hbar = \omega_r' a^\dagger a + +\omega_{a}\frac{\sigma_z}{2}$$ when written in terms of $\omega_r'$, or alternatively $$H_{\text{eff}}/\hbar = \omega_ra^\dagger a + \omega_a'\frac{\sigma_z}{2}$$ when written in terms of $\omega_a'$.
Let's look at our effective Hamiltonian from two different perspectives. First, assume we know the state of the qubit (atom) with exquisite accuracy. 526
We can then evaluate $\omega_r'$ and find that the resonator frequency depends on the qubit state (since $\sigma_z$ gives $\pm1$ depending on whether we are in the state $|0\rangle$ or $|1\rangle$). Now if we drive our resonator with photons in the vicinity of $\omega_r'$, we would find that the accumulated phase shift of these drive photons depends on the precise value of $\omega_r'$ (and thus the state of the qubit). One might say that the drive photons are 'dispersed' in a way that depends on the state of the qubit. Now let's switch things up and suppose instead that we know the photon number in the resonator very well. In that case, I would claim that we could evaluate $\omega_a'$ (which depends on $a^\dagger a$ and thus the photon number) and would find that the qubit frequency had shifted due to the presence of photons in the resonator.
At the end of the day, these are just two different ways of looking at the same term. For more details on dispersive measurement in superconducting qubits, I encourage you to read the first three chapters of <PERSON> PhD thesis. 976