I probably worded it badly, but I would consider the people that wanted to hold onto the status quo as the "conservatives" of the time and place.  Also, if you consider the civil rights movement ongoing,  conservatives get called racist and worse if they push a color blind society and try to end affirmative action.