I think the  problem is that for many, they do back the sentiment.  In traveling around the south in the early 90's, you would still hear folks proclaim "The south will rise again", which at the time struck me as pretty weird.  Today, you see people backing the sentiments in their trying to explain the Civil War away as a states rights issue, while completely ignoring the articles of secession that, to a state, mentioned slavery.  The statues are just one more manifestation of that idea of backing the sentiment.

That being said, I'm not advocating for the statues to come down, that is a local issue that needs to be taken up in the localities impacted.  What I am saying is that those statues are a monument to traitors and that many of those who support keeping them also, at least to a degree, support the ideas that those traitors stood for.