In an electromagnetic wave, both the electric field and magnetic field are oscillating but in different directions; by convention the "polarization" of light refers to the polarization of the electric field.