How does Illya overpower Alter Saber's Excalibur with her traced version? A traced Noble Phantasm should be one rank below that of the original. 

How was Illya able to overcome the difference in power?

Clearly she traced Excalibur and not Caliburn.
It would appear that Archerko Illya traced Excalibur and overloaded it with prana turning it into a Broken Phantasm. Which raises the weapon's rank by one. 

One would assume that the two Excaliburs would cancel each other out, but Saber Alter, unlike the typical Saber only has an Magic Resistance of Rank B since her Alignment is Lawful Evil, which might make her more vulnerable to the effects of Excalibur.