Wind's effect on light rider? I live in Florida which is flat and often windy. I am a light rider, weighing about 125 pounds (57 kilograms).

How significant is the wind push around for someone my size?

My wheels are Reynolds Strike SLG carbon clinchers 622mm 

There are lots of cross and head winds here, had me down to 13.6 mph today for a bit! I am a triathlete and have a big race coming up in Panama City, Florida, and ready for some aero benefits. I normally ride in the 18-20mph zone.
This is pretty simple. All things being equal, aerodynamic drag is a function of the relative wind velocity squared. Doubling the effective wind speed increases the drag force by a factor of four. Head winds could easily account for the loss in speed you are experiencing. Your body weight isn't a factor in dealing with head winds. Indeed if you are relative small, your drag will be lower than a larger rider. Weight is much more of a factor on hills, but in Florida that shouldn't be much of an issue.