real-time reporting tools for recommended in aggregating readership future and social activities per document 7 Summer 2012 – 25 Media (video and Data vary from digital media Hadoop and Pig; Personalized Netflix Movie million subscribers, 4 properties) and to user rankings, user Cassandra; Teradata recommender systems Service million ratings per rankings profiles, and media using logistic/linear day, 3 million continually properties for content-based regression, elastic nets, searches per day, 1 updated recommendations matrix factorization, billion hours clustering, LDA, streamed in June association rules, 2012; Cloud storage decision trees, – 2 petabytes in June and others; streaming 2013 video delivery 8 45 billion web pages Real-time updating Multiple media MapReduce + Bigtable; Crawling; searching, Web Search total, 500 million and real-time Dryad + Cosmos; including topic-based photos uploaded responses to PageRank; final step searches; ranking; each day, 100 hours queries essentially a recommending of video uploaded to recommender engine YouTube each minute 9 Terabytes up to Can be real time Must work for all data Hadoop, MapReduce, Robust backup Business petabytes for recent changes open source, and/or Continuity and vendor proprietary such Disaster Recovery as AWS, Google Cloud Within a Cloud Services, and Microsoft Eco-System 10 — Needs to become Event-based — Distributed event analysis Cargo Shipping real time, currently identifying problems updated at events 11 500,000 material Ongoing increase Many datasets with no National programs No broadly applicable Materials Data types in 1980s, much in new materials standards (Japan, Korea, and analytics for growth since then China), application areas Manufacturing (EU nuclear program), proprietary systems (Granta, etc.)
real-time recommended in future reporting tools for aggregating readership and social activities per document 7 Netflix Movie Service Summer 2012 – 25 million subscribers, 4 million ratings per day, 3 million searches per day, 1 billion hours streamed in June 2012; Cloud storage – 2 petabytes in June 2013 Media (video and properties) and rankings continually updated Data vary from digital media to user rankings, user profiles, and media properties for content-based recommendations Hadoop and Pig; Cassandra; Teradata Personalized recommender systems using logistic/linear regression, elastic nets, matrix factorization, clustering, LDA, association rules, decision trees, and others; streaming video delivery 8 Web Search 45 billion web pages total, 500 million photos uploaded each day, 100 hours of video uploaded to YouTube each minute Real-time updating and real-time responses to queries Multiple media MapReduce + Bigtable; Dryad + Cosmos; PageRank; final step essentially a recommender engine Crawling; searching, including topic-based searches; ranking; recommending 9 Business Continuity and Disaster Recovery Within a Cloud Eco-System Terabytes up to petabytes Can be real time for recent changes Must work for all data Hadoop, MapReduce, open source, and/or vendor proprietary such as AWS, Google Cloud Services, and Microsoft Robust backup 10 Cargo Shipping — Needs to become real time, currently updated at events Event-based — Distributed event analysis identifying problems 11 Materials Data for Manufacturing 500,000 material types in 1980s, much growth since then Ongoing increase in new materials Many datasets with no standards National programs (Japan, Korea, and China), application areas (EU nuclear program), proprietary systems (Granta, etc.) No broadly applicable analytics