Big Data Data Source Terabase-scale sequencing data from JGI, Characteristics (distributed/centralized) subsurface and surface hydrological and biogeochemical data from a variety of sensors (including dense geophysical datasets) experimental data from field and lab analysis Volume (size) Velocity (e.g. real time) Variety Data crosses all scales from genomics of the (multiple datasets, mashup) microbes in the soil to watershed The SFA requires the synthesis of diverse and disparate field, laboratory, and simulation datasets across different semantic, spatial, and temporal scales through GEWaSC. Such datasets will be generated by the different research areas and include simulation data, field data (hydrological, geochemical, geophysical), ‘omics data, and data from laboratory experiments. Variability (rate of Simulations and experiments change) Big Data Science Veracity (Robustness Each of the sources samples different properties (collection, Issues) and Quality with different footprints extremely heterogeneous. – curation, Each of the soruces has different levels of analysis, uncertainty and precision associated with it. In action) addition, the translation across scales and domains introduces uncertainty as does the data mining. Data quality is critical. Visualization Visualization is crucial to understanding the data. Data Types Described in above. “Variety” Data Analytics Data mining, data quality assessment, across datasets, reduced model development, statistics, quality assessment, data fusion, etc. Big Data Specific Translation across diverse and large datasets that cross domains and Challenges (Gaps) scales. Big Data Specific Challenges in Mobility Field experiment data taking would be improved by access to existing data and automated entry of new data via mobile devices. Security & Privacy Requirements Highlight issues for A wide array of programs in the earth sciences are working on challenges generalizing this use that cross the same domains as this project. case (e.g. for ref. architecture)
Big Data Characteristics Data Source (distributed/centralized) Terabase-scale sequencing data from JGI, subsurface and surface hydrological and biogeochemical data from a variety of sensors (including dense geophysical datasets) experimental data from field and lab analysis Volume (size) Velocity (e.g. real time) Variety (multiple datasets, mashup) Data crosses all scales from genomics of the microbes in the soil to watershed The SFA requires the synthesis of diverse and disparate field, laboratory, and simulation datasets across different semantic, spatial, and temporal scales through GEWaSC. Such datasets will be generated by the different research areas and include simulation data, field data (hydrological, geochemical, geophysical), ‘omics data, and data from laboratory experiments. Variability (rate of change) Simulations and experiments Big Data Science (collection, curation, analysis, action) Veracity (Robustness Issues) and Quality Each of the sources samples different properties with different footprints – extremely heterogeneous. Each of the soruces has different levels of uncertainty and precision associated with it. In addition, the translation across scales and domains introduces uncertainty as does the data mining. Data quality is critical. Visualization Visualization is crucial to understanding the data. Data Types Described in “Variety” above. Data Analytics Data mining, data quality assessment, across datasets, reduced model development, statistics, quality assessment, data fusion, etc. Big Data Specific Challenges (Gaps) Translation across diverse and large datasets that cross domains and scales. Big Data Specific Challenges in Mobility Field experiment data taking would be improved by access to existing data and automated entry of new data via mobile devices. Security & Privacy Requirements Highlight issues for generalizing this use case (e.g. for ref. architecture) A wide array of programs in the earth sciences are working on challenges that cross the same domains as this project.