The federation represents thousands of rank and file PSNI officers.
Its chairman, Mark Lindsay, has said confidence in the office of the ombudsman is at an all-time low.
A spokesperson for the ombudsman said the comments were "disappointing" and "at odds with the feedback from police officers who had been investigated".
Mr Lindsay told the federation's annual conference that "reform is not so much desirable as essential".
He also said the PSNI is in a state of crisis because of reducing budgets and officer numbers.
Criticism of the office of the ombudsman by the federation is not new.
In his speech on Wednesday, Mr Lindsay said the federation accepted the need for the independent investigation of complaints against police officers.
But, with Ombudsman Dr Michael Maguire among the audience at the La Mon House Hotel, he suggested some feel they are victims of a "witch hunt".
"It gives me no pleasure to say that, unfortunately, there is very little officer confidence in relation to the operation of the ombudsman's office," he said.
"The credibility of the office is also seriously questioned."
The federation chairman highlighted the ongoing PSNI investigation into the theft and unlawful disclosure of documents from the ombudsman's office.
He also cited the discovery of a gun and ammunition in a filing cabinet belonging to a member of staff in the office last year.
"There is an obvious need for an independent avenue of appeal, whereby officers will have meaningful redress against malicious and inept investigations," he said.
"The ombudsman is with us today and I say to him that this isn't about you, but rather the legislative architecture of the system you preside over.
"If your office conducts itself with the high standards you demand of police officers, then surely you have nothing to fear from such accountability."
Mr Lindsay said the level of assaults against police officers by members of the public is "unacceptably high" and called for tougher action to protect them.
He told the conference that eight out of 10 PSNI officers who responded to a survey said they had been victims of a physical or verbal assault during the past year.
It's understood that 14% of PSNI officers took part in the survey.
The federation has called for the introduction of "spitguards" to prevent people in custody spitting at officers.
He also warned of "dire consequences" for the PSNI because of reducing budgets and officer numbers.
"We're now beyond the point of warning of a crisis. Right now, we're in the middle of a crisis," Mr Lindsay said.
"What's needed now is for our politicians from all sides to make a stand against this madness of continuing austerity.
"They must make policing one of their main priorities and campaign as never before to get the government to re-think its disastrous policy."
The Police Federation has called for an independent appeals process to allow challenges to investigations reports by the Police Ombudsman.