Links tell us about lexical and semantic Web content
<|reference_start|>Links tell us about lexical and semantic Web content: The latest generation of Web search tools is beginning to exploit hypertext link information to improve ranking\cite{Brin98,Kleinberg98} and crawling\cite{Menczer00,Ben-Shaul99etal,Chakrabarti99} algorithms. The hidden assumption behind such approaches, a correlation between the graph structure of the Web and its content, has not been tested explicitly despite increasing research on Web topology\cite{Lawrence98,Albert99,Adamic99,Butler00}. Here I formalize and quantitatively validate two conjectures drawing connections from link information to lexical and semantic Web content. The clink-content conjecture states that a page is similar to the pages that link to it, i.e., one can infer the lexical content of a page by looking at the pages that link to it. I also show that lexical inferences based on link cues are quite heterogeneous across Web communities. The link-cluster conjecture states that pages about the same topic are clustered together, i.e., one can infer the meaning of a page by looking at its neighbours. These results explain the success of the newest search technologies and open the way for more dynamic and scalable methods to locate information in a topic or user driven way.<|reference_end|>
@article{menczer2001links,
    title={Links tell us about lexical and semantic Web content},
    author={Filippo Menczer},
    journal={arXiv preprint arXiv:cs/0108004},
    year={2001},
    archivePrefix={arXiv},
    eprint={cs/0108004},
    primaryClass={cs.IR cs.DL}
}