Background The Continuous Performance Test (CPT) has emerged as the most commonly administered measure of sustained attention, but use of discrepant versions reduces the ability of researchers and clinicians to accurately draw cross-study conclusions. In an effort to standardize use of the CPT, this study compared four versions of the Identical Pairs CPT for their reliability and ability to discriminate between patients with schizophrenia and healthy volunteers. The relationship of performance on the different versions of the CPT with measures of psychopathology, functioning, and other aspects of cognition was also examined. Methods Performance on the 2-digit, 3-digit, 4-digit, and Shapes Identical Pairs CPT was assessed at three test sessions over five weeks, during which subjects also completed the Brief Assessment of Cognition in Schizophrenia (BACS) and questionnaires assessing psychopathology and functioning. Results All four CPTs discriminated between patients with schizophrenia and healthy volunteers, but there were no statistical differences in sensitivity among the versions. The 3-digit CPT showed non-statistical advantages in that it had high test–retest reliability, low potential for a ceiling effect, and a very low rate of false alarms. There were also moderate correlations between CPT performance and performance of the BACS subtests, but no significant correlations between CPT performance and measures of psychopathology and functioning. Conclusions While all versions of the CPT tested here had good psychometric characteristics, the 3-digit CPT-IP has some advantages in repeated measures studies such as clinical trials.