Example DATA MINING –

Visualization

Challenge & Selection     Data mining is very powerful, but it faces many challenges during its  implementation. The challenges could be related to performance, data,  methods and techniques used etc. The data mining process becomes successful  when the challenges or issues are identified correctly and sorted out properly.  Data mining normally leads to serious issues in terms of data security, privacy  and governance. For example, when a retailer analyzes the purchase details, it  reveals information about buying habits and preferences of customers without  their permission.    Although data mining is still in its infancy, companies in a wide range of  industries - including retail, finance, heath care, manufacturing transportation,  and aerospace - are already using data mining tools and techniques to take  advantage of historical data. By using pattern recognition technologies and  statistical and mathematical techniques to sift through warehoused  information, data mining helps analysts recognize significant facts,  relationships, trends, patterns, exceptions and anomalies that might otherwise  go unnoticed.    It is difficult to represent the information in an accurate and easy-to-understand  way to the end user. The input data and output information being complex,  very effective and successful data visualization techniques need to be applied to  make it successful.   There are many more challenges in data mining in addition to the above  specified issues. More challenges get revealed as the actual data mining process  starts and the success of data mining lies in overcoming all these challenges       Application     Data Mining is primarily used today by companies with a strong consumer focus  — retail, financial, communication, and marketing organizations, to “drill down”  into their transactional data and determine pricing, customer preferences and

product positioning, impact on sales, customer satisfaction and corporate  profits. With data mining, a retailer can use point-of-sale records of customer  purchases to develop products and promotions to appeal to specific customer  segments.     The final step of knowledge discovery from data is to verify that the patterns  produced by the data mining algorithms occur in the wider data set. Not all  patterns found by data mining algorithms are necessarily valid. It is common for  data mining algorithms to find patterns in the training set which are not present  in the general data set. This is called overfitting. To overcome this, the  evaluation uses a test set of data on which the data mining algorithm was not  trained. The learned patterns are applied to this test set, and the resulting  output is compared to the desired output. For example, a data mining algorithm  trying to distinguish "spam" from "legitimate" emails would be trained on  a training set of sample e-mails. Once trained, the learned patterns would be  applied to the test set of e-mails on which it had not been trained. The accuracy  of the patterns can then be measured from how many e-mails they correctly  classify. Several statistical methods may be used to evaluate the algorithm, such  as ROC curves.     For the activity, I will divide the stakeholders into groups of 5-6 participants  each representing a different organization or area of interest. Each group  receives a problem and situation on data mining graphs that they can write on  directly or use sticky notes to make comments. I give them two-three questions  to answer, such as: 1) What key elements are missing from the situation? 2)  What would you change about the data mining problems to make it more  accurately reflect your problem space? 3) What key insights do you have about  the relationships between the entities in your problem space? I give them  approximately 20 minutes to discuss the questions and mark up the graphic.  Then I have each group report out on their answers to those three questions  while pointing to the mark-ups on their poster. As each group reports out, I  record commonalities in the insights of each group, and then additional insights  that come from the group discussion on an additional copy of the graphic. After  the session, I digitize each group’s poster as an artifact, create a view that  combines the shared and new insights from the group discussion, and then  create a revised graphic that incorporates their feedback and proposed changes  that they agreed to as a group.

Insight & Approach     Businesses which have been slow in adopting the process of data mining are  now catching up with the others. Extracting important information through  the process of data mining is widely used to make critical business decisions.  In the coming decade, we can expect data mining to become as ubiquitous  as some of the more prevalent technologies used today. Some of the key  data mining trends for the future include

Data mining is used to discover patterns and relationships in the data in order to  help make better business decisions. Data mining can help spot sales trends,  develop smarter marketing campaigns, and accurately predict customer loyalty.  Specific uses of data mining include:

 Market segmentation - Identify the common characteristics of customers

who buy the same products from your company.

 Customer churn - Predict which customers are likely to leave your company

and go to a competitor.

 Fraud detection - Identify which transactions are most likely to be

fraudulent.

 Direct marketing - Identify which prospects should be included in a mailing

list to obtain the highest response rate.

 Interactive marketing - Predict what each individual accessing a Web site is

most likely interested in seeing.

 Market basket analysis - Understand what products or services are

commonly purchased together; e.g., beer and diapers.

 Trend analysis - Reveal the difference between a typical customer this

month and last.

In the short-term, the results of data mining will be in profitable, if mundane,  business related areas. Micro-marketing campaigns will explore new niches.  Advertising will target potential customers with new precision.

In the medium term, data mining may be as common and easy to use as e-mail.  We may use these tools to find the best airfare to New York, root out a phone  number of a long-lost classmate, or find the best prices on lawn mowers.

The long-term prospects are truly exciting. Imagine intelligent agents turned loose  on medical research data or on sub-atomic particle data. Computers may reveal  new treatments for diseases or new insights into the nature of the universe.


Example DATA MINING – Visualization Challenge & Selection Data mining is very powerful , but it faces many challenges during its implementation . The challenges could be related to performance , data , methods and techniques used etc . The data mining process becomes successful when the challenges or issues are identified correctly and sorted out properly . Data mining normally leads to serious issues in terms of data security , privacy and governance . For example , when a retailer analyzes the purchase details , it reveals information about buying habits and preferences of customers without their permission . Although data mining is still in its infancy , companies in a wide range of industries - including retail , finance , heath care , manufacturing transportation , and aerospace - are already using data mining tools and techniques to take advantage of historical data . By using pattern recognition technologies and statistical and mathematical techniques to sift through warehoused information , data mining helps analysts recognize significant facts , relationships , trends , patterns , exceptions and anomalies that might otherwise go unnoticed . It is difficult to represent the information in an accurate and easy - to - understand way to the end user . The input data and output information being complex , very effective and successful data visualization techniques need to be applied to make it successful . There are many more challenges in data mining in addition to the above specified issues . More challenges get revealed as the actual data mining process starts and the success of data mining lies in overcoming all these challenges Application Data Mining is primarily used today by companies with a strong consumer focus — retail , financial , communication , and marketing organizations , to “ drill down ” into their transactional data and determine pricing , customer preferences and product positioning , impact on sales , customer satisfaction and corporate profits . With data mining , a retailer can use point - of - sale records of customer purchases to develop products and promotions to appeal to specific customer segments . The final step of knowledge discovery from data is to verify that the patterns produced by the data mining algorithms occur in the wider data set . Not all patterns found by data mining algorithms are necessarily valid . It is common for data mining algorithms to find patterns in the training set which are not present in the general data set . This is called overfitting . To overcome this , the evaluation uses a test set of data on which the data mining algorithm was not trained . The learned patterns are applied to this test set , and the resulting output is compared to the desired output . For example , a data mining algorithm trying to distinguish " spam " from " legitimate " emails would be trained on a training set of sample e - mails . Once trained , the learned patterns would be applied to the test set of e - mails on which it had not been trained . The accuracy of the patterns can then be measured from how many e - mails they correctly classify . Several statistical methods may be used to evaluate the algorithm , such as ROC curves . For the activity , I will divide the stakeholders into groups of 5 - 6 participants each representing a different organization or area of interest . Each group receives a problem and situation on data mining graphs that they can write on directly or use sticky notes to make comments . I give them two - three questions to answer , such as : 1 ) What key elements are missing from the situation ? 2 ) What would you change about the data mining problems to make it more accurately reflect your problem space ? 3 ) What key insights do you have about the relationships between the entities in your problem space ? I give them approximately 20 minutes to discuss the questions and mark up the graphic . Then I have each group report out on their answers to those three questions while pointing to the mark - ups on their poster . As each group reports out , I record commonalities in the insights of each group , and then additional insights that come from the group discussion on an additional copy of the graphic . After the session , I digitize each group ’s poster as an artifact , create a view that combines the shared and new insights from the group discussion , and then create a revised graphic that incorporates their feedback and proposed changes that they agreed to as a group . Insight & Approach Businesses which have been slow in adopting the process of data mining are now catching up with the others . Extracting important information through the process of data mining is widely used to make critical business decisions . In the coming decade , we can expect data mining to become as ubiquitous as some of the more prevalent technologies used today . Some of the key data mining trends for the future include Data mining is used to discover patterns and relationships in the data in order to help make better business decisions . Data mining can help spot sales trends , develop smarter marketing campaigns , and accurately predict customer loyalty . Specific uses of data mining include :  Market segmentation - Identify the common characteristics of customers who buy the same products from your company .  Customer churn - Predict which customers are likely to leave your company and go to a competitor .  Fraud detection - Identify which transactions are most likely to be fraudulent .  Direct marketing - Identify which prospects should be included in a mailing list to obtain the highest response rate .  Interactive marketing - Predict what each individual accessing a Web site is most likely interested in seeing .  Market basket analysis - Understand what products or services are commonly purchased together ; e.g. , beer and diapers .  Trend analysis - Reveal the difference between a typical customer this month and last . In the short - term , the results of data mining will be in profitable , if mundane , business related areas . Micro - marketing campaigns will explore new niches . Advertising will target potential customers with new precision . In the medium term , data mining may be as common and easy to use as e - mail . We may use these tools to find the best airfare to New York , root out a phone number of a long - lost classmate , or find the best prices on lawn mowers . The long - term prospects are truly exciting . Imagine intelligent agents turned loose on medical research data or on sub - atomic particle data . Computers may reveal new treatments for diseases or new insights into the nature of the universe .