Previous studies report that acquired prosopagnosia is frequently associated with topographic disorientation. Whether this is associated with a specific anatomic subtype of prosopagnosia, how frequently it is seen with the developmental variant, and what specific topographic function is impaired to account for this problem are not known. We studied ten subjects with acquired prosopagnosia from either occipitotemporal or anterior temporal (AT) lesions and seven with developmental prosopagnosia. Subjects were given a battery of topographic tests, including house and scene recognition, the road map test, a test of cognitive map formation, and a standardized self-report questionnaire. House and/or scene recognition were frequently impaired after either occipitotemporal or AT lesions in acquired prosopagnosia. Subjects with occipitotemporal lesions were also impaired in cognitive map formation: an overlap analysis identified right fusiform and parahippocampal gyri as a likely correlate. Only one subject with acquired prosopagnosia had mild difficulty with directional orientation on the road map test. Only one subject with developmental prosopagnosia had difficulty with cognitive map formation, and none were impaired on the other tests. Scores for house and scene recognition correlated most strongly with the results of the questionnaire. We conclude that topographic disorientation in acquired prosopagnosia reflects impaired place recognition, with a contribution from poor cognitive map formation when there is occipitotemporal damage. Topographic impairments are less frequent in developmental prosopagnosia.