NERSC. More at between observation NCO, parallel other HPC centers and simulation NetCDF N/A N/A From 'omics of the PFLOWTran, Data mining, data microbes in the soil to postgres, HDF5, quality assessment, DOE-BER watershed Akuna, NEWT, etc cross-correlation Subsurface across datasets, Biogeochemistry 49 From observation to reduced model simulation development, statistics, quality assessment, data fusion N/A Streaming data from Flux data needs to be EddyPro, Custom Data mining, data ~150 towers in merged with analysis software, R, quality assessment, DOE-BER AmeriFlux and over biological, python, neural cross-correlation AmeriFlux and 500 towers distributed disturbance, and other networks, Matlab across datasets, data FLUXNET 50 globally collecting ancillary data assimilation, data Networks flux measurements interpolation, statistics, quality assessment, data fusion 4 TB a year for a city Streaming data from Tuple-based: R/Matlab, Weka, Forecasting models, with 1.4M sensors like million(s) of sensors Timeseries, database Hadoop. GIS based machine learning Consumption Los Angeles rows; Graph-based: visualization models, time series forecasting in Network topology, analysis, clustering, 51 Smart Grids customer motif detection, connectivity; Some complex event semantic data for processing, visual normalization network analysis Picture Book of Big Data Use Cases Figure 1: Cargo Shipping (Section 2.3.6) Scenario Figure 2: Pathology Imaging/digital pathology (Section 2.5.2) Examples of 2-D and 3-D pathology images. Figure 3: Pathology Imaging/digital pathology (Section 2.5.2) Architecture of Hadoop-GIS, a spatial data warehousing system over MapReduce to support spatial analytics for analytical pathology imaging. Figure 4: DataNet Federation Consortium DFC (Section 2.7.1) iRODS architecture Figure 5: Catalina Real-Time Transient Survey (CRTS): a digital, panoramic, synoptic sky survey (Section 2.8.1) One possible schematic architecture for a cyber-infrastructure for time domain
NERSC. More at other HPC centers between observation and simulation NCO, parallel NetCDF 49 DOE-BER Subsurface Biogeochemistry N/A N/A From 'omics of the microbes in the soil to watershed From observation to simulation PFLOWTran, postgres, HDF5, Akuna, NEWT, etc Data mining, data quality assessment, cross-correlation across datasets, reduced model development, statistics, quality assessment, data fusion 50 DOE-BER AmeriFlux and FLUXNET Networks N/A Streaming data from ~150 towers in AmeriFlux and over 500 towers distributed globally collecting flux measurements Flux data needs to be merged with biological, disturbance, and other ancillary data EddyPro, Custom analysis software, R, python, neural networks, Matlab Data mining, data quality assessment, cross-correlation across datasets, data assimilation, data interpolation, statistics, quality assessment, data fusion 51 Consumption forecasting in Smart Grids 4 TB a year for a city with 1.4M sensors like Los Angeles Streaming data from million(s) of sensors Tuple-based: Timeseries, database rows; Graph-based: Network topology, customer connectivity; Some semantic data for normalization R/Matlab, Weka, Hadoop. GIS based visualization Forecasting models, machine learning models, time series analysis, clustering, motif detection, complex event processing, visual network analysis Picture Book of Big Data Use Cases Figure 1: Cargo Shipping (Section 2.3.6) Scenario Figure 2: Pathology Imaging/digital pathology (Section 2.5.2) Examples of 2-D and 3-D pathology images. Figure 3: Pathology Imaging/digital pathology (Section 2.5.2) Architecture of Hadoop-GIS, a spatial data warehousing system over MapReduce to support spatial analytics for analytical pathology imaging. Figure 4: DataNet Federation Consortium DFC (Section 2.7.1) iRODS architecture Figure 5: Catalina Real-Time Transient Survey (CRTS): a digital, panoramic, synoptic sky survey (Section 2.8.1) One possible schematic architecture for a cyber-infrastructure for time domain