Appendix B: Summary of Key Properties Information related to five key properties was extracted from each use case. The five key properties were three Big Data characteristics (volume, velocity, and variety), software related information, and associated analytics. The extracted information is presented in Table B-1. Table B-1: Use Case Specific Information by Key Properties Use Case Volume Velocity Variety Software Analytics 1 380 TB Static for 75 years Scanned documents Robust archival storage None for 75 years Census 2000 and 2010 2 Hundreds of Data loaded in Unstructured and structured Custom software, Crawl/index, search, NARA: Search, terabytes, and batches, so bursty data: textual documents, commercial search ranking, predictive search; Retrieve, growing emails, photos, scanned products, commercial data categorization Preservation documents, multimedia, databases (sensitive, confidential, social networks, web sites, etc.); personally databases, etc. identifiable information (PII) detection and flagging 3 Approximately 1 PB Variable, field data Strings and numerical data Hadoop, Spark, Hive, R, Recommendation systems, Statistical Survey streamed SAS, Mahout, continued monitoring Response continuously, Allegrograph, MySQL, Improvement Census was ~150 Oracle, Storm, million records BigMemory, Cassandra, transmitted Pig 4 — — Survey data, other Hadoop, Spark, Hive, R, New analytics to create Non-Traditional government administrative SAS, Mahout, reliable information from Data in Statistical data, web-scraped data, Allegrograph, MySQL, non-traditional disparate Survey Response wireless data, e-transaction Oracle, Storm, sources Improvement data, (potentially) social BigMemory, Cassandra, media data and positioning Pig data from various sources 5 — Real time — Hadoop RDBMS XBRL Fraud detection Cloud Eco-System for Finance 6 15 TB presently, Currently Hadoop PDF documents and log files Hadoop, Scribe, Hive, Standard libraries for Mendeley growing about 1 TB batch jobs of social network and client Mahout, Python machine learning and per month scheduled daily, activities analytics, LDA,
Appendix B: Summary of Key Properties Information related to five key properties was extracted from each use case. The five key properties were three Big Data characteristics (volume, velocity, and variety), software related information, and associated analytics. The extracted information is presented in Table B-1. Table B-1: Use Case Specific Information by Key Properties Use Case Volume Velocity Variety Software Analytics 1 Census 2000 and 2010 380 TB Static for 75 years Scanned documents Robust archival storage None for 75 years 2 NARA: Search, Retrieve, Preservation Hundreds of terabytes, and growing Data loaded in batches, so bursty Unstructured and structured data: textual documents, emails, photos, scanned documents, multimedia, social networks, web sites, databases, etc. Custom software, commercial search products, commercial databases Crawl/index, search, ranking, predictive search; data categorization (sensitive, confidential, etc.); personally identifiable information (PII) detection and flagging 3 Statistical Survey Response Improvement Approximately 1 PB Variable, field data streamed continuously, Census was ~150 million records transmitted Strings and numerical data Hadoop, Spark, Hive, R, SAS, Mahout, Allegrograph, MySQL, Oracle, Storm, BigMemory, Cassandra, Pig Recommendation systems, continued monitoring 4 Non-Traditional Data in Statistical Survey Response Improvement — — Survey data, other government administrative data, web-scraped data, wireless data, e-transaction data, (potentially) social media data and positioning data from various sources Hadoop, Spark, Hive, R, SAS, Mahout, Allegrograph, MySQL, Oracle, Storm, BigMemory, Cassandra, Pig New analytics to create reliable information from non-traditional disparate sources 5 Cloud Eco-System for Finance — Real time — Hadoop RDBMS XBRL Fraud detection 6 Mendeley 15 TB presently, growing about 1 TB per month Currently Hadoop batch jobs scheduled daily, PDF documents and log files of social network and client activities Hadoop, Scribe, Hive, Mahout, Python Standard libraries for machine learning and analytics, LDA,