Prosecutors said he had maintained a "criminal relationship" with officials at a construction conglomerate.
His lawyer said he was innocent and called the arrest a "spectacle".
Dozens of politicians and Petrobras executives have been arrested and sentenced after a two-year anti-corruption investigation.
The investigation, called the "Car Wash operations" has unveiled a massive corruption scheme at Petrobras.
In a news conference, prosecutors said Mr Palocci had favoured construction giant Odebrecht while in government.
They said he had facilitated contracts with Petrobras and helped the group obtain credit from a state bank in exchange for  donations to the governing Workers' Party.
Mr Palocci is alleged to have continued to intermediate bribes even after he left office.
Last week, police temporarily detained another former finance minister, Guido Mantega, also on charges related to the Car Wash probe.
He was released hours later because his wife was about to undergo surgery.
Earlier this month a judge ruled that former President Luiz Inacio Lula da Silva would face trial on corruption charges over the Petrobras scandal.
Mr Palocci served as a finance minister under Lula and as chief of staff to his successor, Ms Rousseff, who was impeached last month for manipulating the budget.

Taiwan's Tingyi - which operates the instant noodle and beverage Master Kong brand in China - struck the deal with PepsiCo earlier this month.
A worker taking part in the action told the BBC staff were "taking leave together" to defend their legal rights.
A spokeswoman from PepsiCo said that their contracts would be honoured.
In a brief statement, a Pepsi spokeswoman in Beijing said: "PepsiCo is a responsible employer. We have high commitments on our staffers' rights."
"We are holding positive communications with our employees."
Tingyi struck a deal with Pepsi on 4 November to buy out PepsiCo China's bottling division.
Media reports in China suggest that the deal has aroused widespread discontent, and workers from five factories in Chongqing, Chengdu, Fuzhou, Lanzhou and Nanchang have decided to walk out.
There are a total of 24 bottling factories in China run by PepsiCo, which employ some 20,000 workers.
PepsiCo China did not say how many workers were taking industrial action, but a Sina Weibo page said to be put up by workers from the southern city of Fuzhou said 1,100 people there had taken leave to join the protest.
An operator for another page on Sina Weibo - China's equivalent of Twitter - which claims to be representing workers in the central Chengdu factory told the BBC that some 500 staff members were "on leave", equivalent to half of the factory's workforce.
Chinese newspaper the Economic Observer reported that under the tie-up PepsiCo workers' contracts will be terminated and they will have to negotiate new terms with the joint venture.
Many of the protesters believe this will be disadvantageous to them, according to the report.
But the PepsiCo China statement said: "Our deal with Master Kong is yet to receive government approval.  Once it is approved, labour contracts that belong to the bottling cluster will still be honoured."
The Chengdu PepsiCo factory worker told the BBC via Sina Weibo that the protesters were defending their legal rights.
"To us, Pepsi is not just a job that we keep in order to feed our family, it is a career that we all work hard to pursue."
Former Brazilian Finance Minister Antonio Palocci has been arrested in connection with a long-running inquiry into corruption at oil giant Petrobras.
[NEXT_CONCEPT]
Workers from at least five PepsiCo bottling factories in China are taking part in protests against a takeover deal.