41 Terabytes/year Data updated Big data uniform Custom analysis based Pattern recognition, EISCAT 3D (current), 40 PB/year continuously with on flat file data demanding correlation incoherent starting ~2022 real-time test storage routines, high-level scatter radar analysis and batch parameter extraction system full analysis 42 Low volume (apart Mainly real-time Six separate projects with R and Python Data assimilation, ENVRI from EISCAT 3D data streams common architecture for (Matplotlib) for (statistical) analysis, data Environmental given above), one infrastructure, data very visualization, custom mining, data extraction, Research system EPOS ~15 diverse across projects software for scientific modeling and Infrastructure TB/year processing simulation, scientific workflow 43 Around 1 PB Data taken in Raw data, images with final Matlab for custom Custom signal processing CReSIS Remote (current) increasing ~two-month layer data used for science raw data processing, to produce radar images Sensing by 50–100 TB per missions including custom image that are analyzed by image mission, future test analysis and processing software, processing to find layers expedition ~1 PB then later batch GIS as user interface each processing 44 110 TB raw data and Data come from Image and annotation files ROI_PAC, GeoServer, Process raw data to get UAVSAR Data 40 TB processed, aircraft and so GDAL, images that are run Processing plus smaller samples incrementally tools; through image processing added, data moving to clouds tools and accessed from occasionally get GIS reprocessed: new processing methods or parameters 45 MERRA collection Periodic updates Many applications to SGE Univa Grid Engine Federation software NASA LARC/GSFC (below) represents every six months combine MERRA reanalysis Version 8.1, iRODS iRODS most of total data, data with other reanalyses Version 3.2 and/or other smaller and observational data such 3.3, IBM GPFS Version collections as CERES 3.4, Cloudera Version 4.5.2-1 46 480 TB from MERRA Increases at ~1 Applications to combine Cloudera, iRODS, Climate MERRA Analytic TB/month MERRA reanalysis data with Amazon AWS (CAaaS) Services other re-analyses and observational data
41 EISCAT 3D incoherent scatter radar system Terabytes/year (current), 40 PB/year starting ~2022 Data updated continuously with real-time test analysis and batch full analysis Big data uniform Custom analysis based on flat file data storage Pattern recognition, demanding correlation routines, high-level parameter extraction 42 ENVRI Environmental Research Infrastructure Low volume (apart from EISCAT 3D given above), one system EPOS ~15 TB/year Mainly real-time data streams Six separate projects with common architecture for infrastructure, data very diverse across projects R and Python (Matplotlib) for visualization, custom software for processing Data assimilation, (statistical) analysis, data mining, data extraction, scientific modeling and simulation, scientific workflow 43 CReSIS Remote Sensing Around 1 PB (current) increasing by 50–100 TB per mission, future expedition ~1 PB each Data taken in ~two-month missions including test analysis and then later batch processing Raw data, images with final layer data used for science Matlab for custom raw data processing, custom image processing software, GIS as user interface Custom signal processing to produce radar images that are analyzed by image processing to find layers 44 UAVSAR Data Processing 110 TB raw data and 40 TB processed, plus smaller samples Data come from aircraft and so incrementally added, data occasionally get reprocessed: new processing methods or parameters Image and annotation files ROI_PAC, GeoServer, GDAL, tools; moving to clouds Process raw data to get images that are run through image processing tools and accessed from GIS 45 NASA LARC/GSFC iRODS MERRA collection (below) represents most of total data, other smaller collections Periodic updates every six months Many applications to combine MERRA reanalysis data with other reanalyses and observational data such as CERES SGE Univa Grid Engine Version 8.1, iRODS Version 3.2 and/or 3.3, IBM GPFS Version 3.4, Cloudera Version 4.5.2-1 Federation software 46 MERRA Analytic Services 480 TB from MERRA Increases at ~1 TB/month Applications to combine MERRA reanalysis data with other re-analyses and observational data Cloudera, iRODS, Amazon AWS Climate (CAaaS)