Do autopilot headings consider the earth's shape? How do you set correct heading while cruising across latitudes?

On a flat map, point A might be to point B at, say, heading 060 (intentionally no cardinal headings taken for example, as they don't create this situation). But on a curved surface, like spherical earth, the heading doesn't remain constant- it keeps changing as you move away from or toward the equator, sending aircraft miles off-target over long distances, as a result of pole-bound spiraling of the path.

Is there a formula to determine the optimum heading at the outset (even though it may not 'seem' correct), thereby, reaching the desired point B on the globe at the end of the cruise?
Yes, almost all modern computer navigation systems take this into account.

The track between two points along the "spherical" earth is called the great-circle track. Except for a N/S heading (or a E/W heading at the equator), the heading will vary along the track. Ed Williams has compiled a formulary for great circle navigation here, mostly derived from the Haversine formula. The formula you are interested in is "Course between points" for calculating the initial heading to fly from point A to point B.