Eh - humans generally have a hard time rating things on a 5 star rating system. That&#x27;s mainly because the discrete intervals between 1 and 5 stars aren&#x27;t well defined so everyone will have a different opinion of what 3 stars &quot;mean&quot;. I encountered this when I ran a rating system in Mechanical Turk. If you ask turkers to do something arbitrary like &quot;rate the quality of this picture&quot; it takes quite a number of ratings on the same image to get an accurate consensus.<p>It would be interesting to see if you can replicate experiment but replace the 5 stars with 5 yes&#x2F;no questions regarding their performance and see if those match up