You bet. With your same thinking, remove the statue of liberty and MLK for starters.
Any historical memorials, take them down. All of them.
Guarantee somebody will not agree with their historical actions these monuments were erected for, so remove them all.