  Recent observations of Kepler multi-planet systems have revealed a number of
systems with planets very close to second-order mean motion resonances (MMRs,
with period ratio $1:3$, $3:5$, etc.) We present an analytic study of resonance
capture and its stability for planets migrating in gaseous disks. Resonance
capture requires slow convergent migration of the planets, with sufficiently
large eccentricity damping timescale $T_e$ and small pre-resonance
eccentricities. We quantify these requirements and find that they can be
satisfied for super-Earths under protoplanetary disk conditions. For planets
captured into resonance, an equilibrium state can be reached, in which
eccentricity excitation due to resonant planet-planet interaction balances
eccentricity damping due to planet-disk interaction. We show that this
"captured" equilibrium can be overstable, leading to partial or permanent
escape of the planets from the resonance. In general, the stability of the
captured state depends on the inner to outer planet mass ratio $q=m_1/m_2$ and
the ratio of the eccentricity damping times. The overstability growth time is
of order $T_e$, but can be much larger for systems close to the stability
threshold. For low-mass planets undergoing type I (non-gap opening) migration,
convergent migration requires $q \lesssim 1$, while the stability of the
capture requires $q\gtrsim 1$. These results suggest that planet pairs stably
captured into second-order MMRs have comparable masses. This is in contrast to
first-order MMRs, where a larger parameter space exists for stable resonance
capture. We confirm and extend our analytical results with $N$-body
simulations, and show that for overstable capture, the escape time from the MMR
can be comparable to the time the planets spend migrating between resonances.

Earth and Planetary Astrophysics