Use Case Requirements Summary Data Data Security & Lifecycle Use Case Sources Capabilities Consumer Privacy Management Others Transformation 1. large -- 1. large -- 1. Title 13 data 1. long-term -- document format centralized preservation of Census from a storage (storage) data as-is for 75 2010 and centralized years 2000 storage 2. long-term preservation at the bit-level 3. curation 1 process including format transformation 4. access and analytics processing after 75 years 5. needs to make sure no data loss 1. distributed 1. crawl and 1. large data 1. high relevancy 1. security policy 1. pre-process for 1. mobile search data sources index from storage and high recall virus scan with similar NARA: 2. large data distributed data 2. various from search 2. file format interfaces/results Search, storage sources storages such as 2. high accuracy identification from desktop Retrieve, 3. bursty data 2. various NetApps, from 3. indexing range from GB to analytics Hitachi, categorization of 4. categorize hundreds of TB processing Magnetic tapes records records 4. wide variety of including 3. various data formats ranking, data storages such as including categorization, NetApps, unstructured and detect PII data Hitachi, 2 structured data 3. pre-processing Magnetic tapes 5. distributed of data data sources in 4. long-term different clouds preservation management fo large varied datasets. 5. hugh amount of data with high relevancy and recall. 1. data size 1. analytics are 1. software 1. data 1. improving 1. high veracity 1. mobile access approximately required for includes Hadoop, visualization for recommendation on data and Statistical one petabyte recommendation Spark, Hive, R, data review, systems that systems must be Survey systems, SAS, Mahout, operational reduce costs and very robust. The Response continued Allegrograph, activity and improve quality semantic monitoring and MySQL, Oracle, general analysis. while providing integrity of general survey Storm, It continues to confidentiality conceptual 3 improvement. BigMemory, evolve. safeguards that metadata Cassandra, Pig are reliable and concerning what publically exactly is auditable. measured and the 2. both resulting limits confidential and of inference secure all data. remain a
Use Case Requirements Summary Use Case Data Sources Transformation Capabilities Data Consumer Security & Privacy Lifecycle Management Others 1 Census 2010 and 2000 1. large document format from a centralized storage -- 1. large centralized storage (storage) -- 1. Title 13 data 1. long-term preservation of data as-is for 75 years 2. long-term preservation at the bit-level 3. curation process including format transformation 4. access and analytics processing after 75 years 5. needs to make sure no data loss -- 2 NARA: Search, Retrieve, 1. distributed data sources 2. large data storage 3. bursty data range from GB to hundreds of TB 4. wide variety of data formats including unstructured and structured data 5. distributed data sources in different clouds 1. crawl and index from distributed data sources 2. various analytics processing including ranking, data categorization, detect PII data 3. pre-processing of data 4. long-term preservation management fo large varied datasets. 5. hugh amount of data with high relevancy and recall. 1. large data storage 2. various storages such as NetApps, Hitachi, Magnetic tapes 1. high relevancy and high recall from search 2. high accuracy from categorization of records 3. various storages such as NetApps, Hitachi, Magnetic tapes 1. security policy 1. pre-process for virus scan 2. file format identification 3. indexing 4. categorize records 1. mobile search with similar interfaces/results from desktop 3 Statistical Survey Response 1. data size approximately one petabyte 1. analytics are required for recommendation systems, continued monitoring and general survey improvement. 1. software includes Hadoop, Spark, Hive, R, SAS, Mahout, Allegrograph, MySQL, Oracle, Storm, BigMemory, Cassandra, Pig 1. data visualization for data review, operational activity and general analysis. It continues to evolve. 1. improving recommendation systems that reduce costs and improve quality while providing confidentiality safeguards that are reliable and publically auditable. 2. both confidential and secure all data. 1. high veracity on data and systems must be very robust. The semantic integrity of conceptual metadata concerning what exactly is measured and the resulting limits of inference remain a 1. mobile access