Question: Imagine a car taking a curve with constant speed. Assuming no friction or air resistance all we would have would be centripetal force. As the car's front wheels straighten to exit the curve the centripetal force gets smaller and smaller. Yet, the car's velocity's direction follows the wheels' direction. So there has to be some force making that happen, right?

What force is that? Where does it come from? How is it calculated?

Answer: 
When a car turns, the wheels are set to rotate in a direction other than forward. The normal rolling action of the wheel will cause the car to turn. This is the path of least friction - in the absence of friction (such as on ice), the car would continue to go forward.

When the car has started to turn, the occupants and other loose things would continue to head straight forward, under the inertia. But friction and physical restraints would cause these to stop their motion. (Excess motion could cause toppling). 

Centrifugal force is not a real thing. It is simply an indicated force required to keep something in circular motion. In saying that centrifugal force is matched by gravity (in an orbit), the centrifugal force is a condition needed to be applied to keep the object in orbit, and the gravity is the substance of the condition.
You may have inadvertently begun to answer your own question. The centripetal force decreases as the car decreases it's rate of turning to exit the curve.

If you think about it, it is actually really intuitive. The car wants to go in a straight line, and if you are driving on ice and the wheels stop turning and you no longer have friction on the wheels, the car will slide in a straight line approximately tangent to the curve it was just following. The only thing keeping the car on a curve is the centripetal force, so when that force decreases, the cars velocity starts to straighten out until there is no longer a centripetal force and you are travelling in a straight line.

In other words, there is no force causing the exiting of a curve, it is precisely the lack of force.