document: Smart Miner : a new framework for mining large scale web usage data In this paper , we propose a novel framework called Smart-Miner for web usage mining problem which uses link information for producing accurate user sessions and frequent navigation patterns . Unlike the simple session concepts in the time and navigation based approaches , where sessions are sequences of web pages requested from the server or viewed in the browser , Smart Miner sessions are set of paths traversed in the web graph that corresponds to users ' navigations among web pages . We have modeled session construction as a new graph problem and utilized a new algorithm , Smart-SRA , to solve this problem efficiently . For the pattern discovery phase , we have developed an efficient version of the Apriori-All technique which uses the structure of web graph to increase the performance . From the experiments that we have performed on both real and simulated data , we have observed that Smart-Miner produces at least 30 % more accurate web usage patterns than other approaches including previous session construction methods . We have also studied the effect of having the referrer information in the web server logs to show that different versions of Smart-SRA produce similar results . Our another contribution is that we have implemented distributed version of the Smart Miner framework by employing Map\/Reduce Paradigm . We conclude that we can efficiently process terabytes of web server logs belonging to multiple web sites by our scalable framework .
doc_bio_tags: O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O