content_image/998490.jpg
encountered by the text analytics as mentioned in “Challenges in text analytics” on page 62. Even if you can reach 100% perfection in text analytics, the numbers produced might still not reflect reality. 

For example, suppose that you want to analyze customer contact records that involve a specific product failure. In this case, you want to know the total number of units sold that are experiencing the failure or the total number of people who have encountered the product failure. It is not practical to expect that all of the customers might report the failure. Only some of them might call the customer contact center. If the failure is serious, many people might report the problem. If the failure is not serious, only a few people might call in. In the customer contact center, the agents might inadvertently leave out critical information when recording the problem as described by the customers. 

Because of all of these factors, even after you make a significant effort to improve the accuracy of your text analytics, you cannot determine the true number of customers who experienced the problem nor the number of products that caused the problem. Figure 4-2 illustrates this point. 

<img src='content_image/998490.jpg'>

Even though the frequency counts might not be as reliable as you want, their deviations often lead to valuable insights. For example, 10% of the calls this month on product A were associated with problem X, and last month only 3% of the calls on product A were associated with problem X. In this case, the change or deviation  is an indicator of a potential problem that is worth further investigation. That is assuming that no surge in selling product A in recent months nor more demand in using product A in recent months has occurred. Making the example more concrete, let us use the paint pealing complaints on toy series. If complaints on paint peeling for toy X series have been increasing from June 2013, the company should investigate to see why there is a change in the complaints frequencies. 

