Thinking about the one single scenario where it goes on forever, as if it were an event unto itself, is misleading imo -- by that reasoning, in that scenario, you would win an infinite amount even if you were only getting paid $1 on each flip.  But the EV is clearly not infinite for that game.  The increasingly huge rare payouts are essential to the EV calc.