  Small quasi-two-dimensional (2D) dust clusters consisting of three to eleven
particles are formed in an argon plasma under varying rf power. Their normal
modes are investigated through their mode spectra obtained from tracking
thermal motion of the particles. Detailed coupling patterns between their
horizontal and vertical modes are detected for particle numbers up to seven and
discrete instabilities are found for dust clusters with particle number greater
than nine, as predicted in previous theory on ion-flow induced mode coupling in
small clusters. The instabilities are proven to be induced by resonance between
coupled horizontal and vertical normal modes.

Plasma Physics