What'd pegasi have? Weather control? Unicorns could just give themselves cloud-walking and move clouds themselves. It didn't matter if we were here, it didn't matter if we had that magic ourselves naturally, because they'd just replace us.