Its Secretary General Bheki Ntshalintshali said he was no longer the "right person" to lead the country.
Anti-apartheid struggle veterans have also called on the African National Congress (ANC) to recall the president.
Mr Zuma has been under growing pressure following a major cabinet reshuffle which included the sacking of respected Finance Minister Pravin Gordhan.
That led to South Africa's credit rating being cut to junk status putting more pressure on a troubled economy.
The Congress of South African Trade Unions (Cosatu), a key part of the governing alliance, says it has 1.8 million members.
It forms part of what is called the Tripartite Alliance along with the the African National Congress (ANC) and the South African Communist Party (SACP).
The SACP has also called on Mr Zuma to go.
Cosatu General Secretary of Bheki Ntshalintshali told a media briefing that the union's decision was driven by Mr Zuma's failure to consult it before making changes to his cabinet.
He termed the president's leadership as "inattentive, negligent... and disruptive".
He added that the organisation was not concerned about Mr Gordhan's sacking because he was, like his predecessors, "not a friend of the workers".
"We will support the new minister where necessary and fight with him where necessary," he added.
Mr Ntshalintshali also criticised ratings agency S&P's decision to downgrade South Africa to junk status saying the union views it as political interference.
ANC veterans, who include former high commissioners, ministers and many respected anti-apartheid activists, also told a media briefing in Johannesburg that the ANC should do "the honourable thing and recall the president", especially after the party's integrity commission advised that he should resign.
President Jacob Zuma is certainly politically weaker today than he was over the weekend. Cosatu's call for him to step down is a crushing blow to the beleaguered head of state.
The workers' union federation had been a reliable backer of Mr Zuma against unrelenting calls for his removal.
Focus now shifts to the National Working Committee (NWC) of the ANC which is currently meeting.
However, regardless of the outcome of that meeting, deep divisions within the body will entrench positions between the pro and anti-Zuma factions.
Mr Zuma is due to step down in 2019 at the end of his second five-year term as president.
Last week, Deputy President Cyril Ramaphosa called the sacking of Mr Gordhan "totally unacceptable".
Later, in a speech at the weekend, which has been interpreted as a public broadside against Mr Zuma, he called for a renewal of the country and criticised "greedy and corrupt people".
Former President Kgalema Motlanthe also said that it was difficult for Mr Zuma to command respect after the constitutional court found him in breach of the law when he failed to repay government money spent on his private home.
But President Zuma's obituary has been written many times before only for him to rise from the ashes, says the BBC's Milton Nkosi in Johannesburg..
Despite the reported scandals that have dogged his administration, several attempts to remove Mr Zuma have foundered.
Meanwhile, new Finance Minsiter Malusi Gigaba has been working to reassure South Africans about the state of the economy.
Mr Gigaba told a media briefing that Monday's downgrade to junk status by ratings agency S&P was a setback, but that people should not be despondent.
"I'm not saying it's easy to get out of a rating downgrade, yet I remain confident," he added.
He said he would lead a meeting with ratings agencies Fitch and Moody's.
South Africa's powerful trades union federation Cosatu has called for President Jacob Zuma to step down.