Previous studies have demonstrated impairments in attention, memory and executive functions in euthymic bipolar patients (BP) as well as their unaffected first-degree relatives, albeit in an attenuated form. Subsequently, cognitive deficits are discussed as a possible endophenotype of bipolar disorder. However, recent studies showed that only a subgroup of BP shows cognitive impairments. The aim of the present study was to investigate cognitive functioning in relatives compared to BP, to find out if the differentiation in a cognitive deficit vs. non-deficit subgroup is valid for relatives of BP, too. Therefore, the performance of 27 unaffected relatives of BP, 27 euthymic BP and 27 HC were compared using a neuropsychological test battery. The results showed that BP exhibited a reduced psychomotor speed and deficits in working memory compared to relatives and HC. Relatives performed significantly slower (psychomotor speed) as compared to HC (p = 0.024); performance in the other test measures lie between BP and HC. Furthermore, a detailed evaluation of the data indicated that only subgroups of BP and relatives exhibited cognitive impairments in the implemented tests. However, the deficit and non-deficit groups did not differ in sociodemographic and clinical variables from each other, possibly due to the small sample size. In conclusion, our results suggest that reduced psychomotor speed could serve as a potential endophenotype for bipolar disorder which should be investigated along the developmental trajectory of this disorder, also to examine whether abnormalities therein precede onset of the first mood episode. Furthermore, the division of relatives into subgroups aids in the identification of stable trait markers and high-risk bipolar groups and could enable early prevention strategies. As to that more research using distinct and homogeneous subgroups is necessary.