In order to address these limitations in a practical way , we conducted a small user study where we asked four judges ( graduate students from the Faculty of Information Technology at Monash University ) to assess the responses generated by our system ( Marom and Zukerman 2007a ) .
evaluation of a largescale email response system
Y Marom I Zukerman
Further , user-based evaluations of the output produced by our system require the subjects to read relatively long request -- response e-mails , which quickly becomes tedious . In order to address these limitations in a practical way , we conducted a small user study where we asked four judges ( graduate students from the Faculty of Information Technology at Monash University ) to assess the responses generated by our system ( Marom and Zukerman 2007a ) . Our judges were instructed to position themselves as help-desk customers who know that they are receiving an automated response , and that such a response is likely to arrive quicker than a response composed by an operator .
In order to address these limitations in a practical way , we conducted a small user study where we asked four judges ( graduate students from the Faculty of Information Technology at Monash University ) to assess the responses generated by our system ( @@CITATIONa ) .