Are there rules for how many riders can be in a World Tour team? Pro teams take nine riders to a grand tour, and fewer riders to other races. Of course, not every rider will ride every race.

Is there an upper limit to the total number of riders a team can have at any given time?

Alternatively, is there another mechanism to keep the relative strength between the teams in check?
The UCI does have rules for the maximum number of riders. The base max is 28, but you can get extra spots for having new pros on your roster, up to 2 extra spots. The rule comes from section "2.15.110 Riders" of the UCI regulations. This cycling blog has a good explanation of it written up: http://inrng.com/2011/08/max-team-size-uci-rules/