It may still end up with some sort of a litigation, if  the culprit would  produce credible evidence that the was not at fault. In other words, a court case. He/she can only be punished if found guilty. As any court case, this might take a long time, not to mention an appeal. Suppose it succeeds, so what would be the likely punishment? In case of losing the mandate, someone else must be  elected, which is also a lengthy and expensive process. Who would pay?
I don't see how it would be different from the present system.