This is an excellent point and it&#x27;s one of the ways we tried to reduce the interview subjectivity at my last employer.<p>Don&#x27;t get me wrong: in the end, an interview is highly subjective, but when five people are evaluating someone, you want to have some way of calibrating their expectations so they are at least close to being on the same page.<p>It helped to even out the process. If Tim says the candidate was a 3 out of 5 on multithreading concepts, I know what questions he could have been asked (there may be 6 to choose from on the list) and what quality of answers he gave (typical candidate responses are shown, in increasing order of &quot;quality&quot;).<p>Did it help get us better developers? I don&#x27;t think so. In the end we concluded that our biggest problem wasn&#x27;t screening good from bad candidates -- we usually made pretty good hires. The problem was getting good candidates to apply in the first place.