David Warner and Steve Smith both hit centuries as Australia reached 371-6 in their 50 overs at Kingsmead.
But an unbroken seventh-wicket stand of 107 by David Miller (118 not out) and Andile Phehlukwayo (42 not out) saw the Proteas home with four balls to spare.
South Africa now have an unassailable 3-0 lead in the five-match series.
Their 372-6 eclipsed India's score of 362-1 - also against Australia - in 2013.
The highest successful one-day international run chase was also made by South Africa, who hit 438-9 - against Australia - in Johannesburg in 2006.
Eight of the top 10 successful ODI run chases have now been achieved against Australia.
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.

The Celtic left-back was handed the role in Scotland's crucial World Cup qualifier against Slovenia.
And Tierney could continue there against England at Hampden on 10 June.
"It's up to the gaffer, whatever he thinks is best and whatever he thinks the team that is going to give us the best chance of winning," said Tierney.
Scotland had been left short at right-back after Alan Hutton retired from international football and Callum Paterson suffered cruciate ligament damage.
Derby County winger Ikechi Anya had impressed at right-back against England and Canada.
But, as Hull City's Andy Robertson revealed after the 1-0 win over Slovenia, head coach Gordon Strachan decided to move one of his three left-backs over to the left.
Media playback is not supported on this device
Rangers' Lee Wallace missed out through injury, Robertson was chosen on the left and Strachan showed his faith in 19-year-old Tierney for a debut in the right-back role.
"I have been training there all week, so I had a feeling I'd be playing there," said Tierney, who was winning his third Scotland cap.
"The manager and the coaching staff have been giving me pointers. I enjoyed it."
Strachan said before the game that his side needed three points to retain hopes of qualifying for the 2018 finals in Russia.
Instead, the victory moved the Scots two points off second-top Slovakia and gave them a fighting chance of making a play-off.
It was a big occasion for Tierney's Celtic team-mate, Stuart Armstrong, to win his first cap and the midfielder responded by setting up Chris Martin's 88th-minute winning goal four days before his 25th birthday.
Asked about his role in the winner, the former Dundee United player said: "It feels good, but what feels better is to have contributed in a positive way to three points, which we needed to put us in a very good position in the group," he said.
"Kech [Ikechi Anya] has made a really good run on the outside just to drag that defender and Chris has held a really good position in the box and has done brilliantly to turn and finish."
Armstrong also collected the man-of-the-match award, while Strachan declared it the best Scotland debut he could remember.
Clutching his cap, he said: "It's a proud moment to hold this.
"It's a proud moment for my family. To cap that off with three points is great."
Strachan was widely expected to quit as team boss had they failed to win, but Armstrong thought the side responded well to such pressure.
"We had trained well all week, there was a good atmosphere, and it's important sometimes to stay positive when things aren't going the way you want them to," he said.
"And I think we responded really well."
South Africa completed the second-highest run chase in one-day international history as they beat Australia by four wickets in Durban.
[NEXT_CONCEPT]
Kieran Tierney is prepared to continue as Scotland's auxiliary right-back after coming through a high-stakes experiment with flying colours.