Facial affect processing (FAP) deficits in schizophrenia (SZ) and bipolar disorder (BD) have been widely reported; although effect sizes vary across studies, and there are limited direct comparisons of the two groups. Further, there is debate as to the influence of both psychotic and mood symptoms on FAP. This study aimed to address these limitations by recruiting groups of psychosis patients with either a diagnosis of SZ or BD and comparing them to healthy controls (HC) on a well validated battery of four FAP subtests: affect discrimination, name affect, select affect and match affect. Overall, both groups performed more poorly than controls in terms of accuracy. In SZ, this was largely driven by impairments on three of the four subtests. The BD patients showed impaired performance specifically on the match affect subtest, a task that had a high cognitive load. FAP performance in the psychosis patients was correlated with severity of positive symptoms and mania. This study confirmed that FAP deficits are a consistent finding in SZ that occur independent of task specific methodology; whilst FAP deficits in BD are more subtle. Further work in this group is needed to replicate these results.