Prime Minister Tony Blair has apologised after stinging criticism of the government's flagship tax credits.
Tax credits are designed to help working families
Mismanagement of the scheme, designed to benefit working parents on low incomes, had "caused hardship or distress" to some, Mr Blair conceded.
One-third of claimants were overpaid and pushed into poverty when asked to repay, Citizens Advice reported.
And the Parliamentary Ombudsman accused Paymaster General Dawn Primarolo of not giving MPs a clear picture.
Both reports call for overpayments to be written off.
In an emergency statement to MPs, Ms Primarolo said the government was already writing off overpayments which are attributable to a department error.
She said that she had asked HM Revenue & Customs (HMRC) to suspend the recovery of overpayments where there is a dispute between the HMRC and the tax credit claimant.
CAB's damning report - based on 150,000 cases handled by the charity, which runs the Citizens Advice Bureaux - said that HMRC had "failed to live up to its own standards of information, clarity and efficiency of service" in the administration of tax credits.
One of the chief gripes reported to the CAB was overpayment, which topped £500 in more than half the cases.
The Office for National Statistics recently reported that of £13.5bn paid out last year, £1.9bn consisted of overpayments. CAB said the "huge overpayments" were due to HMRC system failings.
The charity called on the government to give an amnesty to people who had received tax credit overpayments, instead of trying to claim them back.
Meanwhile, in her report, Parliamentary Ombudsman Ann Abraham accused Ms Primarolo of failing to give accurate information when questioned about overpayments in the House of Commons in February.
Ms Abraham also said she was unable to say how many overpayments were due to government mistakes and how many were down to delays in claimants reporting a change in circumstances.
But her report echoed CAB's call for an amnesty on the recovery of overpayments, particularly those caused by official errors in the first two years of the scheme.
"This would be a sensible and proportionate response to the situation and would give much-needed relief to people who, in many cases, have been caused considerable distress and hardship," she added.
Not in "collapse"
Ms Primarolo said the system, which has benefited more than six million families and 10 million children, was not in "complete collapse".
She stressed she had been clear about the extent of the problems. "Before that date and since that date I have continued to acknowledge that there were problems...I have made it clear in parliament that there are issues with the IT system and the administrative system," she said.
The Shadow Conservative Chancellor, George Osborne, called on Ms Primarolo to appear before the House of Commons to explain the handling of tax credit overpayments.
"It is a scandalous that a system meant to help people is causing so much hardship and distress, to the point where people are having to borrow money from loan sharks to repay money sent to them in error," Mr Osborne said.
David Laws, the Liberal Democrat Shadow Work and Pensions Secretary, said that the reports showed that the design of the system was "flawed."
Mr Laws added that the call to write off overpayments was "sensible and logical".