The purpose of this research was to study individual differences among students using rating scales (“rubrics”) to assess a complex performance. More precisely, we tried to determine which characteristics of a student’s judgment in terms of accuracy, severity and confidence have an impact on writing achievement. Accuracy was found to be the best predictor of writing achievement. Several important differences in the students’ ability to use rating scales were found among students with learning difficulties, gifted and talented students. Despite the fact that girls usually had better marks in writing, no differences were found between boys and girls as far as accuracy, severity and confidence were concerned. Multiple regressions on francophone and anglophone students’ results indicated that rating scales contributed in the same way to writing achievement in both groups. Degree of confidence in one’s assessment of a peer’s writing was significantly lower among francophones. No significant improvement in judgment accuracy was found from grade 5 to grade 8. The research concludes by stressing the importance of socio-affective aspects of a student’s judgment in the use of rating scales.