The town of Charlottesville voted for Hillary almost 18,000 to 3,000.  It's very possible most of the counter-protestors were at home.  On the other hand, Charlottesville lies in a sea of red counties.  I suspect most of the "Unite the Right" group were from out-of-time.