Peter Hain has apologised for "the embarrassment caused by poor administration" but insisted that he wants to get on with his cabinet jobs.
Mr Hain has been under pressure over the failure to declare on time more than £100,000 donated to his failed bid to become Labour's deputy leader.
The work and pensions and Welsh secretary said the idea he had attempted "to hide anything is absurd".
But Tory MP David Davies called Mr Hain's statement "slightly arrogant".
Mr Hain, who read out the text of his statement to reporters outside his constituency home in Neath, said he had already "unreservedly apologised" for failing to register all the donations at the right time.
But he said all those who had donated money to his campaign were "eligible and legally entitled to do so".
Mr Hain also said the senior civil servant at the Department for Work and Pensions was "satisfied" that none of the donations had breached the ministerial code.
Pressure has been growing on Mr Hain after he disclosed on Thursday that he had failed to declare £103,000 in donations - on top of the £82,000 he did declare.
Mr Hain, who finished fifth of six in the race to succeed John Prescott as Labour's deputy leader, blamed his government responsibilities for distracting him from the running of his campaign.
There were also questions over the role of the Progressive Policies Forum think-tank, which channelled money to his deputy leadership bid.
The little-known think-tank was set up three months after the launch of Mr Hain's campaign and counts John Underwood, who was closely involved in financing the Hain campaign, as a trustee.
Mr Underwood has said that the £26,000 in donations and £25,000 interest-free loan from PPF to the Hain campaign were "entirely permissible".
The BBC understands two donors were not told their money was going towards his campaign but Mr Underwood said: "I can confirm that, as several of these donors have now said publicly, they were content for their contributions to PPF subsequently to be donated to Peter Hain's campaign."
In his statement Mr Hain said that it was only after the end of the campaign that "the extent of outstanding debt became apparent to me".
He said: "I was not involved in establishing the Progressive Policies Forum but it was prepared and able to assist in making donations to the campaign and did so."
Mr Hain said he was also "now making immediate arrangements" to repay the £25,000 interest-free loan from the PPF.
Mr Hain's failure to declare such large amounts has led to inquiries being launched by the Electoral Commission and the Parliamentary Commissioner for Standards.
In his statement, which he read out without taking questions, Mr Hain said: "I apologise for the embarrassment caused by poor administration and lack of early and clear disclosure.
"However I make no apology for being a committed candidate in Labour's deputy leadership campaign or for raising the funds necessary to run a modern campaign.
"I am more than happy for the inquiries by the Electoral Commission and the Parliamentary Commissioner for Standards to take their course and meanwhile I will get on with my cabinet jobs."
However Conservative MP David Davies said: "I think it's slightly arrogant to just say, 'I have done nothing wrong, it's a bit of forgetfulness on my part, you know, sorry I forgot about the £100,000 but I'm off to do my two jobs again now'.
"I don't really think that's going to wash with people actually."
Scottish National Party MP Angus MacNeil said: "It's bit strange that we have the donating of money to Peter Hain going through somebody else... The whole world around Labour and Peter Hain is very murky and it's certainly not very clear-cut at all."
Plaid Cymru's Westminster leader, Elfyn Llwyd, said: "He does two very important jobs in cabinet and if he can't get this amount of paperwork right, then the questions must be asked generally about his abilities to carry on in the jobs that he's doing."
Lib Dem leader Nick Clegg told the BBC he could not "distinguish between what appears to be either utter incompetence in the way his campaign was run, or deliberate obfuscation".
A Downing Street spokesman confirmed that the prime minister retained confidence in Mr Hain.