The credibility of witnesses is an important factor in any criminal trial, either for or against the accused. Are you saying you want to take that feature out of our justice system? To do so would be a fundamental change, overturning centuries of legal precedence.

And you're forgetting how the justice system determines witness (or complainant) credibility in the first place: via corroborating evidence. Be it circumstantial or direct, evidence usually reinforces one testimony while eroding or even contradicting a different version. And there's almost always more evidence in criminal cases than simply the recollection of a witness.