The present study deals with the measurement of a specific aspect of writing competency. It is an exploratory study and its goal is to analyse the validity of interpretations done about writing competency with scores coming from answers to multiple-choice items. To do so, items prototypes were developed and tried with a small group of students at the college level. Each examinee score is compared to content analysis of one essay and is explained with a theoretical framework of writing competency. Even if our results are limited to the present study, they constitute an opening to study the use of multiple choice items to assess other aspects of writing competency than the linguistic code.