case (e.g. for ref. architecture) More Information (URLs) Government Operation> Use Case 3: Statistical Survey Response Improvement Use Case Title Statistical Survey Response Improvement (Adaptive Design) Vertical (area) Government Statistical Logistics Author/Company/Email Cavan Capps: U.S. Census U.S. statistical agencies are charged to be the leading authoritative sources about the Actors/Stakeholders people and economy, while honoring privacy and rigorously protecting nation’s and their roles and confidentiality. This is done by working with states, local governments and other responsibilities government agencies. To use advanced methods, that are open and scientifically objective, the statistical Goals agencies endeavor to improve the quality, the specificity and the timeliness of statistics provided while reducing operational costs and maintaining the confidentiality of those measured. Survey costs are increasing as survey response declines. The goal of this work is to use Use Case Description advanced system using data mashed up from several “recommendation techniques” sources and historical survey para-data to drive operational processes in an effort to increase quality and reduce the cost of field surveys. Current Compute(System) Linux systems Solutions Storage SAN and Direct Storage Fiber, 10 gigabit Ethernet, Infiniband 40 gigabit. Networking Hadoop, Spark, Hive, R, SAS, Mahout, Allegrograph, Software MySQL, Oracle, Storm, BigMemory, Cassandra, Pig Survey data, other government administrative data, Big Data Data Source geographical positioning data from various sources. Characteristics (distributed/centralized) Volume (size) For this particular class of operational problem approximately one petabyte. Varies, paradata from field data streamed continuously, Velocity during the decennial census approximately 150 million (e.g. real time) records transmitted. Data is typically defined strings and numerical fields. Data Variety can be from multiple datasets mashed together for (multiple datasets, analytical use. mashup) Varies depending on surveys in the field at a given time. Variability (rate of High rate of velocity during a decennial census. change) Big Data Science Veracity (Robustness Data must have high veracity and systems must be very (collection, curation, Issues, semantics) robust. The semantic integrity of conceptual metadata analysis, concerning what exactly is measured and the resulting action) limits of inference remain a challenge Data visualization is useful for data review, operational Visualization
case (e.g. for ref. architecture) More Information (URLs) Government Operation> Use Case 3: Statistical Survey Response Improvement Use Case Title Statistical Survey Response Improvement (Adaptive Design) Vertical (area) Government Statistical Logistics Author/Company/Email Cavan Capps: U.S. Census Actors/Stakeholders and their roles and responsibilities U.S. statistical agencies are charged to be the leading authoritative sources about the nation’s people and economy, while honoring privacy and rigorously protecting confidentiality. This is done by working with states, local governments and other government agencies. Goals To use advanced methods, that are open and scientifically objective, the statistical agencies endeavor to improve the quality, the specificity and the timeliness of statistics provided while reducing operational costs and maintaining the confidentiality of those measured. Use Case Description Survey costs are increasing as survey response declines. The goal of this work is to use advanced “recommendation system techniques” using data mashed up from several sources and historical survey para-data to drive operational processes in an effort to increase quality and reduce the cost of field surveys. Current Solutions Compute(System) Linux systems Storage SAN and Direct Storage Networking Fiber, 10 gigabit Ethernet, Infiniband 40 gigabit. Software Hadoop, Spark, Hive, R, SAS, Mahout, Allegrograph, MySQL, Oracle, Storm, BigMemory, Cassandra, Pig Big Data Characteristics Data Source (distributed/centralized) Survey data, other government administrative data, geographical positioning data from various sources. Volume (size) For this particular class of operational problem approximately one petabyte. Velocity (e.g. real time) Varies, paradata from field data streamed continuously, during the decennial census approximately 150 million records transmitted. Variety (multiple datasets, mashup) Data is typically defined strings and numerical fields. Data can be from multiple datasets mashed together for analytical use. Variability (rate of change) Varies depending on surveys in the field at a given time. High rate of velocity during a decennial census. Big Data Science (collection, curation, analysis, action) Veracity (Robustness Issues, semantics) Data must have high veracity and systems must be very robust. The semantic integrity of conceptual metadata concerning what exactly is measured and the resulting limits of inference remain a challenge Visualization Data visualization is useful for data review, operational