Background Exposure to adverse childhood experiences (ACE) and cognitive deficits are both prevalent in psychosis. While it has been repeatedly demonstrated that ACE contribute to cognitive dysfunctions, the specific nature of this contribution remains elusive. Recent evidence suggests that types of adversities during critical periods have deleterious effects on brain structures that are important for cognitive functioning. The present study sought to clarify which types of adversities experienced at which time during development aggravate cognitive deficits in psychosis. Methods Exposure to abuse and neglect during childhood and adolescence were retrospectively assessed in N =168 adult individuals with psychotic disorder. Conditioned random forest regression was used to define the importance of type and timing of ACE for predicting domains of the MATRICS Consensus Cognitive Battery (MCCB). Results Significant importance of ACE was determined for 5 out of 7 MCCB domains. Particularly abuse at age 3 contributed to dysfunctional cognitive domains attention, learning, and working memory. Social cognition was related to neglect experienced at 11–12years, and to cumulative ACE. Conclusion Abuse and neglect at periods when children spend substantial time in their families affect cognitive functioning, and hence aggravate dysfunction in psychosis. Results support the neurodevelopmental perspective on psychosis and the diagnostic value of type and timing of ACE.