Timothy Tyrone Foster was convicted of molesting and killing a white 79-year-old retired schoolteacher in 1987.
But the court on Thursday overturned his conviction after ruling that the prosecution had broken the law.
Foster may now face a retrial, 29 years after his death sentence.
A law introduced in 1986 made it illegal in the US to pick jurors based on the colour of their skin.
But the following year all four black members of the potential jury pool in Foster's case were struck from the pool by prosecutors, leaving an all-white jury.
Non race-related reasons were given for striking the black members of the pool, but prosecution notes released to Foster's lawyers in 2006 revealed racial motivations, the Supreme Court said.
The notes show that the prosecution marked the names of black prospective jurors with a "B", highlighted them in green, and circled the word "black" on their juror questionnaires, Reuters news agency reported.
According to Foster's lawyer, Stephen Bright, one handwritten note titled "Definite Nos" listed six people, of whom five were the remaining black prospective jurors, the Associated Press reported.
The sixth was a white woman who made clear she would never impose the death penalty, Mr Bright said.
Chief Justice John Roberts wrote that the notes "plainly belie the state's claim that it exercised its strikes in a 'colour blind' manner".
The eight justices of the Court voted 7-1 in Foster's favour. The sole dissenter was Justice Clarence Thomas, a conservative and the only black member of the court.
Foster, who was 18 at the time of the murder, was accused of breaking into the home of Queen Madge White, breaking her jaw, sexually molesting her and then strangling her, before stealing items from her house.
The US Supreme Court has ruled in favour of a black death-row inmate, finding that state prosecutors in Georgia unlawfully excluded potential black jurors from his trial.