The Wales Audit Office investigated finances at Clydach Community Council and Mawr Community Council, both of which have employed Robert King.
It said Mr King paid his own monthly salary early and overpaid himself at Clydach, and exposed Mawr council to the risk of financial penalties.
Mr King said he had done "nothing inappropriate".
One report examined accounts at Clydach from 2004-05 to 2009-10.
Mr King resigned as clerk in June 2010, although he remained in the post until April 2011.
The Audit Office said Mr King failed to keep proper accounting records, and "due to the poor record keeping of the former clerk, it is unclear as to whether all the income due to the council has been collected and banked".
It also said that "on a large number of occasions the former clerk drew his monthly salary before it was due to be paid.
"In 2005-06 this actually resulted in the former clerk making an overpayment to himself by drawing 13 months' salary.
"The former clerk in many instances also failed to pay the income tax and national insurance contributions to HM Revenue and Customs on time, exposing the council to risk of penalties and interest charges".
A report was also critical of the council, which it said had spent Â£14,676 unnecessarily because of "poor governance and administration".
Auditor Anthony Barrett made 11 recommendations to the council.
But Mr King, who left Clydach of his own choice because he had too much work, said the accounts were "always late" coming back from the auditors.
"The Wales Audit Office had those accounts for three years and did nothing with them," he said.
"I couldn't get on and complete the work."
A separate report examined accounts at Mawr Community Council - where Mr King is still clerk - from 2005-06 to 2009-10.
The audit office said that "the clerk's poor administration of the council's payroll arrangements resulted in overpayments and other errors which have exposed the council to the risk of financial penalties".
Again, the report is critical of the council, and said it spent Â£10,500 unnecessarily due to "poor governance and administration".
But Mr Barrett said he was encouraged the council "is already taking action" to address the concerns raised.
Mr King - who described himself as a "fall guy" - added: "There may have been mistakes, but I have done nothing inappropriate."
"Mawr has always had an internal auditor, who has always checked the books.
"The internal auditor started to nag the audit office because we were sending accounts in and nothing was coming back.
"In the end, it transpired the lady who was supposed to to be looking at our files had emigrated, and nothing was being done."
Clydach Community Council thanked the audit office and said it would adopt its recommendations admitting it had been "naive in trusting established practices inherited from previous administrations".
Two reports have highlighted failings by a clerk at two Swansea Valley community councils.