(7) Changing Analytics Paradigm Data Science – Statistics rigorous causal analysis of carefully sampled data – Data Mining approximate causal analysis of repurposed data carefully sampled – Data Science probabilistic analysis/trending of large selection or even entire dataset – Data Science correlation not necessarily causation – Data Science determine the questions and not the answers – Data Science getting an answer by solving a simpler problem – Data Science - Venn diagram domain, analytics, programming – (8) Changing Analytics Processes While the basic data lifecycle processes remain the same, the order in which they are done can change. The simplest data lifecycle process is: Collection -> data Curation -> information Analysis -> knowledge Action -> wisdom resulting in benefit benefit (putting the knowledge to work) Security Veracity precision/accuracy/timeliness of the data – Provenance a particular kind of metadata about the history (pedigree) of the dataset – (how analyzed, etc) Cleanliness/Quality more data vs more – Obsolescence Filtering MapReduce data query distribution – Grid computing data processing distribution – ??? for when horizontal scalability is insufficient is there a need for concept of processes being coupled (when multiple processes are not independet)? Are there any that cannot be decoupled. Data integration/matching different primary, but secondary fields that can be correlated – Crawlers Bots
(7) Changing Analytics Paradigm – Data Science Statistics – rigorous causal analysis of carefully sampled data Data Mining – approximate causal analysis of repurposed data carefully sampled Data Science – probabilistic analysis/trending of large selection or even entire dataset Data Science – correlation not necessarily causation Data Science – determine the questions and not the answers Data Science – getting an answer by solving a simpler problem Data Science - Venn diagram – domain, analytics, programming (8) Changing Analytics Processes While the basic data lifecycle processes remain the same, the order in which they are done can change. The simplest data lifecycle process is: Collection -> data Curation -> information Analysis -> knowledge Action -> wisdom resulting in benefit benefit (putting the knowledge to work) Security Veracity – precision/accuracy/timeliness of the data Provenance – a particular kind of metadata about the history (pedigree) of the dataset (how analyzed, etc) Cleanliness/Quality – more data vs more Obsolescence Filtering MapReduce – data query distribution Grid computing – data processing distribution ??? for when horizontal scalability is insufficient is there a need for concept of processes being coupled (when multiple processes are not independet)? Are there any that cannot be decoupled. Data integration/matching – different primary, but secondary fields that can be correlated Crawlers Bots