Background Neurocognitive impairments are frequently observed in schizophrenia and major depressive disorder (MDD). However, it remains unclear whether reported neurocognitive abnormalities could objectively identify an individual as having schizophrenia or MDD. Methods The current study included 220 first-episode patients with schizophrenia, 110 patients with MDD and 240 demographically matched healthy controls (HC). All participants performed the short version of the Wechsler Adult Intelligence Scale-Revised in China; the immediate and delayed logical memory of the Wechsler Memory Scale-Revised in China; and seven tests from the computerized Cambridge Neurocognitive Test Automated Battery to evaluate neurocognitive performance. The three-class AdaBoost tree-based ensemble algorithm was employed to identify neurocognitive endophenotypes that may distinguish between subjects in the categories of schizophrenia, depression and HC. Hierarchical cluster analysis was applied to further explore the neurocognitive patterns in each group. Results The AdaBoost algorithm identified individual's diagnostic class with an average accuracy of 77.73% (80.81% for schizophrenia, 53.49% for depression and 86.21% for HC). The average area under ROC curve was 0.92 (0.96 in schizophrenia, 0.86 in depression and 0.92 in HC). Hierarchical cluster analysis revealed for MDD and schizophrenia, convergent altered neurocognition patterns related to shifting, sustained attention, planning, working memory and visual memory. Divergent neurocognition patterns for MDD and schizophrenia related to motor speed, general intelligence, perceptual sensitivity and reversal learning were identified. Conclusions Neurocognitive abnormalities could predict whether the individual has schizophrenia, depression or neither with relatively high accuracy. Additionally, the neurocognitive features showed promise as endophenotypes for discriminating between schizophrenia and depression.