Mr Gordhan will be replaced by Malusi Gigaba, said a statement issued late on Thursday by the preisdent's office.
Earlier this week, President Zuma recalled Mr Gordhan from planned events in the UK and US.
Sfiso Buthelezi will become Deputy Finance Minister, replacing Mcebisi Jonas.
"I have directed the new ministers and deputy ministers to work tirelessly with their colleagues to bring about radical socioeconomic transformation and to ensure that the promise of a better life for the poor and the working class becomes a reality," President Zuma's statement said.
Last October, Mr Gordhan was charged with fraud - but the charges were later dropped.
He has described the allegations as politically motivated.
Mr Gordhan has been seen as standing up to President Zuma in cabinet and has warned against corruption becoming rampant.
The South African Communist Party, an ally of the governing African National Congress, had earlier lodged a formal objection to plans to dismiss Mr Gordhan, who is widely respected internationally.
Many senior ANC figures also opposed the finance minister's removal.

Professional hunter Lee Lightsey said the 15 ft (4.5m) beast was one of the biggest he had come across in 18 years and required a tractor to move it.
Mr Lightsey and hunting guide Blake Godwin discovered the alligator in cattle ponds while supervising a guided hunt on Saturday.
When the creature surfaced about 20ft (6m) away from them, they shot it.
"Although this animal is huge I was not that surprised it existed," Mr Lightsey said. "We have come across lots over the last 20 years that have been only a little smaller.
"But what really drew our attention to this animal was the fact that it seems to have been feasting on the cattle on my farm, because mutilated body parts were found in the water. It was a monster which needed to be removed."
Mr Lightsey's company arranges hunts for alligators, wild boar and turkey on the farm he owns for hunting trips. The largest alligator previously killed was just over 13ft (4m) long.
He charges $10,000 (£7,060; €8,780) to kill an alligator larger than 13ft (4m) and $4,500 (£3,180; €3,955) to kill an alligator between 10ft and 12ft. The animals are mostly killed with a high-powered rifle.
"But always we kill them with the minimum of suffering without allowing them to be injured before they die," he said.
There are hundreds of alligators for hunters between 10ft and 13ft, he says, while animals longer than 13ft are only encountered about once a year.
Mr Lightsey has been commercially hunting alligators since he began "harvesting" them in 1988 and more than 5,000 alligators bigger than 1.5m have been killed since then.
American alligators inhabit the south-east of the US with Florida and Louisiana each reported to have a population of more than one million alligators each.
Southern Florida is reputed to be one of the only places in the world where both alligators and crocodiles live alongside each other in freshwater environments such as rivers, lakes and swamps.
Mr Lightsey says he plans to get the alligator stuffed while the meat will be donated to charity.
South Africa's President Jacob Zuma has dismissed Finance Minister Pravin Gordhan after days of speculation that rocked the country's markets,
[NEXT_CONCEPT]
A hunter in the US has told the BBC how he shot an 800lb (360kg) alligator that was feasting on his farm's cattle.