document: Beyond position bias : examining result attractiveness as a source of presentation bias in clickthrough data Leveraging clickthrough data has become a popular approach for evaluating and optimizing information retrieval systems . Although data is plentiful , one must take care when interpreting clicks , since user behavior can be affected by various sources of presentation bias . While the issue of position bias in clickthrough data has been the topic of much study , other presentation bias effects have received comparatively little attention . For instance , since users must decide whether to click on a result based on its summary ( e.g. , the title , URL and abstract ) , one might expect clicks to favor `` more attractive '' results . In this paper , we examine result summary attractiveness as a potential source of presentation bias . This study distinguishes itself from prior work by aiming to detect systematic biases in click behavior due to attractive summaries inflating perceived relevance . Our experiments conducted on the Google web search engine show substantial evidence of presentation bias in clicks towards results with more attractive titles .
doc_bio_tags: O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O