Jurors could not reach a verdict after four days of deliberations in the case against former North Charleston patrolman Michael Slager, 35.
A bystander filmed the officer opening fire on 50-year-old Walter Scott after he fled an April 2015 traffic stop.
The jury of 11 white people and a black man heard four weeks of testimony.
Charleston County prosecutor Scarlett A Wilson said her office would retry Mr Slager, who is also scheduled to stand trial in federal court next year.
"We hope the federal and state courts will coordinate efforts regarding any future trial dates but we stand ready whenever the court calls," she said in a statement, referring to his federal trial.
The case gained national attention after video emerged appearing to show the moment the officer opened fire on a fleeing Mr Scott.
Mr Scott was fatally shot five times in the back as he tried to run away after he had been pulled over for a broken tail light.
Mr Slager was fired from the police force after the shooting.
He pleaded not guilty to murder, but jurors were also allowed to consider a voluntary manslaughter charge against him as the trial came to a close last week.
The jury announced on Friday they were unable to reach a unanimous verdict, but the judge sent them back to continue deliberations.
One of the jurors wrote Judge Clifton Newman a letter saying he could not "in good conscience approve a guilty verdict" and would not change his mind.
On Monday, a note from the jury said "the majority" of jurors were still undecided.
The panel also submitted a number of questions, including why voluntary manslaughter was added as a possible verdict and whether the definition of self-defence is different for police than for civilians.
Mr Slager had testified that he feared for his life during the encounter.
"Like so many others in our community, I'm deeply disappointed by the jury's inability to reach a unanimous verdict", Charleston Mayor John Tecklenburg said in a statement.
"However, despite that disappointment, I also understand that justice is not always delivered by a single jury, in a single courtroom, on a single day. Justice is often a journey. And the journey to justice in the Michal Slager case is far from over."
The former police officer would have faced 30 years to life in prison on a murder charge while a manslaughter conviction would have carried a sentence of two to 30 years behind bars.
The North Charleston City Council reached a $6.5m (Â£5.1m) settlement with Mr Scott's family in October 2015 over the incident.

However, Wood has been given a warning after his shin made contact with Williams' head as he tried to secure possession for Wales in the 28-25 win over the hosts at Twickenham.
A warning equates to a yellow card.
Three yellow cards in the tournament lead to a disciplinary hearing.
On Sunday, Wood apologised to Williams with a message on Twitter. Wood wrote: "Really relieved to hear you're back on your feet OK mate. Horrible to see you stretchered off because of me. Hope you have a speedy recovery."
Full-back Williams, 24, though, could pay a heavier price for the clash as he can only make a gradual return to play under head injury protocols.
These could see him ruled out of Wales' remaining pool matches against Fiji and Australia.
Williams's potential absence adds to Wales's injury problems with centre Scott Williams (leg) and winger Hallam Amos (shoulder) set to be ruled out of the tournament.
Wales play Fiji in Cardiff on Thursday, before England host Australia at Twickenham on Saturday.
The case of a South Carolina police officer accused of murdering a fleeing black motorist by shooting him dead has ended in a mistrial.
[NEXT_CONCEPT]
England flanker Tom Wood is free to play against Australia after he was not cited for the incident which saw Wales full-back Liam Williams knocked unconscious on Saturday.