Climate Studies from 15 end-to-end simulations groups and between Atmospheric Research storage simulations at NERSC, observation and simulation (NCAR) PIO library and more at other HPC utilities NCL and NCO, centers parallel NetCDF 49 — — From omics of the microbes PFLOWTran, postgres, Data mining, data quality DOE-BER in the soil to watershed HDF5, Akuna, NEWT, assessment, Subsurface hydro-biogeochemistry, from etc. across datasets, Biogeochemistry observation to simulation reduced model development, statistics, quality assessment, data fusion 50 — Streaming data Flux data merged with EddyPro, custom Data mining, data quality DOE-BER from ≈150 towers biological, disturbance, and analysis software, R, assessment, AmeriFlux and in AmeriFlux and other ancillary data Python, neural correlation across datasets, FLUXNET over 500 towers networks, Matlab data assimilation, data Networks distributed globally interpolation, statistics, collecting flux quality assessment, data measurements fusion 51 4 TB/year for a city Streaming data Tuple-based: timeseries, R/Matlab, Weka, Forecasting models, Consumption with 1.4 million from millions of database rows; graph-based: Hadoop; GIS-based machine learning models, forecasting in sensors, such as Los sensors network topology, customer visualization time series analysis, Smart Grids Angeles connectivity; some semantic clustering, motif detection, data for normalization complex event processing, visual network analysis
Climate Studies from 15 end-to-end simulations at NERSC, more at other HPC centers simulations groups and between observation and simulation Atmospheric Research (NCAR) PIO library and utilities NCL and NCO, parallel NetCDF storage 49 DOE-BER Subsurface Biogeochemistry — — From omics of the microbes in the soil to watershed hydro-biogeochemistry, from observation to simulation PFLOWTran, postgres, HDF5, Akuna, NEWT, etc. Data mining, data quality assessment, across datasets, reduced model development, statistics, quality assessment, data fusion 50 DOE-BER AmeriFlux and FLUXNET Networks — Streaming data from ≈150 towers in AmeriFlux and over 500 towers distributed globally collecting flux measurements Flux data merged with biological, disturbance, and other ancillary data EddyPro, custom analysis software, R, Python, neural networks, Matlab Data mining, data quality assessment, across datasets, data assimilation, data interpolation, statistics, quality assessment, data fusion 51 Consumption forecasting in Smart Grids 4 TB/year for a city with 1.4 million sensors, such as Los Angeles Streaming data from millions of sensors Tuple-based: timeseries, database rows; graph-based: network topology, customer connectivity; some semantic data for normalization R/Matlab, Weka, Hadoop; GIS-based visualization Forecasting models, machine learning models, time series analysis, clustering, motif detection, complex event processing, visual network analysis correlation