SUMMERVILLE, S.C. -- A year ago, dozens marched to protest the Confederate flag a white woman flew from her porch in a historically black Southern neighborhood. After someone threw a rock at her porch, she put up a wooden lattice. That was just the start of the building.

Earlier this year, two solid 8-foot high wooden fences were built on either side of Annie Chambers Caddell's modest brick house to shield the Southern banner from view.

Late this summer, Caddell raised a flagpole higher than the fences to display the flag. Then a similar pole with an American flag was placed across the fence in the yard of neighbor Patterson James, who is black.

One hundred and fifty years after the Civil War began about 20 miles away in Charleston Harbor, fights continue over the meaning of the Confederate flag. Some see it as a symbol of slavery and racism; others like Caddell say it's part of their Southern heritage.

"I'm here to stay. I didn't back down and because I didn't cower the neighbors say I'm the lady who loves her flag and loves her heritage," said the 51-year old Caddell who moved into the historically black Brownsville neighborhood in the summer of 2010. Her ancestors fought for the Confederacy.

Last October, about 70 people marched in the street and sang civil rights songs to protest the flag, while about 30 others stood in Caddell's yard waving the Confederate flag.

Opponents of the flag earlier gathered 200 names on a protest petition and took their case to a town council meeting where Caddell tearfully testified that she's not a racist. Local officials have said she has the right to fly the flag, while her neighbors have the right to protest. And build fences.

"Things seemed to quiet down and then the fences started," Caddell said. "I didn't know anything about it until they were putting down the postholes and threw it together in less than a day."

Aaron Brown, the town councilman whose district includes Brownsville, said neighbors raised money for the fences.

"The community met and talked about the situation," he said. "Somebody suggested that what we should do is just go ahead and put the fences up and that way somebody would have to stand directly in front of the house to see the flag and that would mediate the flag's influence."

Caddell isn't bothered by the fences and said they even seem to draw more attention to her house.

"People driving by here because of the privacy fences, they tend to slow down," she said. "If the objective was to block my house from view, they didn't succeed very well."

The Confederate flag remains a sensitive issue in South Carolina.

The battle emblem of the Confederacy had flown on the dome of the Statehouse in Columbia since the Civil War centennial in the 1960s when state lawmakers voted in 2000 to move it to a Confederate monument in front of the building. The National Association for the Advancement of Colored People has waged a tourism boycott on the state since then as it seeks to have the flag removed from the Statehouse grounds.

Caddell, Brown and James all say things have been quiet in Brownsville in recent months.

"She's got a right to do what she wants to do," James said. "That's all I really have to say. She can do what she wants to do in her yard, but I don't share her beliefs."