  Modulation instability (MI) in continuous media described by a system of two
cubic-quintic nonlinear Schr\"odinger equations (NLSE) has been investigated
with a focus on revealing the contribution of the quintic nonlinearity to the
development of MI in its linear and nonlinear stages. For the linear stage we
derive analytic expression for the MI gain spectrum and compare its predictions
with numerical simulations of the governing coupled NLSE. It is found that the
quintic nonlinearity significantly enhances the growth rate of MI and alters
the features of this well known phenomenon by suppressing its time-periodic
character. For the nonlinear stage by employing a localized perturbation to the
constant background we find that the quintic nonlinearity notably changes the
behavior of MI in the central oscillatory region of the integration domain. In
numerical experiments we observe emergence of multiple moving coupled solitons
if the parameters are in the domain of MI. Possible applications of the
obtained results to mixtures of Bose-Einstein condensates and bimodal light
propagation in waveguide arrays are discussed.

Pattern Formation and Solitons