One visiting reverend, John Gillison, said the "devil entered" but failed to "take control of your church".
A black cloth was placed over the chair of Reverend Clementa Pinckney, who was one of those killed by the gunman.
Hundreds of people packed the Emanuel AME church for the service on Sunday morning, amid tight security.
Meanwhile, police are investigating an online post, possibly by suspected gunman Dylann Roof, which may outline his motivation for the attack.
On Sunday, bells at more than a dozen churches rang out simultaneously throughout the state of South Carolina in a sign of solidarity.
"There they were in the house of the Lord, studying your word, praying with one another," Rev John Gillison told the congregation.
"But the devil also entered. And the devil was trying to take charge," he said. But, he added, "the devil cannot take control of your people. And the devil cannot take control of your church".
Volunteers were on hand to give out bottles of water to the congregation, with many members reportedly forced to sit in an overflow part of the church.
Hundreds of people who could not get inside held their own prayers outside the building. Many had travelled from across the country to pay their respects.
South Carolina governor Nikki Haley, who has called for the death penalty for suspect Dylann Roof, attended the service with her family.
There are five chairs at the altar in the Emanuel Church. But the one in the middle was vacant on Sunday, draped instead with a black cloth.
This used to be Reverend Clementa Pinckney's seat. This used to be his congregation. But events on Wednesday night changed all that.
Visiting pastors addressed the regular congregation, who were joined by people of all ages and races, some from across the US.
"The freshness of death comes like a thief in the night," said one of the pastors. But it hasn't stolen the spirit of people here.
The service was uplifting and upbeat - worshippers including Charleston's mayor, and the Governor of South Carolina, Nikki Haley, leapt to their feet at several points, clapping and singing.
One of the songs during the service was Amazing Grace - something people here and across this city have shown in the past few days in remarkable acts of forgiveness and dignity.
Survivors say Mr Roof spent close to an hour in an evening bible study group on Wednesday before opening fire.
A group from the Emanuel AME church met again on Saturday in the room where their friends died on Wednesday.
One of those who attended, Harold Washington, said the cleaners had to cut bullet holes out of the walls so that members of the congregation would not see them on Sunday.
A number of rallies were also staged in South Carolina over the weekend, including a protest in the state capital of Columbia to demand the removal of the Confederate flag from the capitol building.
The flag was a symbol used by southern states in the civil war, when they tried to break away to prevent the abolition of slavery. It is viewed by many as a sign of the white supremacy.
Protesters chanted "take it down" and sang We Shall Overcome, an anthem of the black civil rights movement.
On Friday, South Carolina's Republican state representative Doug Brannon told MSNBC that he planned to introduce legislation to remove the flag.
On Saturday, images emerged on a website showing Dylann Roof posing with the Confederate flag. In others, he is seen burning the US flag and visiting a former slave plantation.
In one image, he is shown staring down the camera while sitting on a chair in camouflage trousers holding a gun.
It is unclear who posted the images on the site, which was found on Saturday.
The website - since taken down - also carried a 2,000-word racist manifesto, the origins of which are also unknown.
The author says Charleston was chosen for the attack because of its history of slavery and its large black population.
Internet records suggest the website's domain was registered in February but it is unclear who was behind it. The FBI says it is investigating the website.
Data from the images show many of them were taken in April and May this year.
The victims of the Charleston shooting
The names of the nine people shot dead at an African-American church in South Carolina have been read out in its first service since Wednesday's attack.