from successor LSST in 2020s 37 Several petabytes Analysis done in Image and simulation data MPI, FFTW, viz New analytics needed to DOE Extreme from Dark Energy batch mode with packages, numpy, analyze simulation results Data from Survey and Zwicky data from Boost, OpenMP, Cosmological Sky Transient Factory, observations and ScaLAPCK, PSQL and Survey simulations > 10 PB simulations MySQL databases, updated daily Eigen, cfitsio, astrometry.net, and Minuit2 38 Petabytes of data 400 images of 1 Images Linux cluster, Oracle Machine learning to find Large Survey Data from Dark Energy GB in size per RDBMS server, optical transients, for Cosmology Survey night Postgres PSQL, large Cholesky decomposition memory machines, for thousands of standard Linux simulations with matrices interactive hosts, of order 1 million on a side GPFS; for simulations, and parallel image storage HPC resources; standard astrophysics reduction software as well as Perl/Python wrapper scripts 39 15 PB of data Data updated Different format for each Grid-based Sophisticated specialized Particle Physics at (experiment and continuously with stage in analysis but data environment with data analysis code LHC Monte Carlo sophisticated uniform within each stage over 350,000 cores followed by basic combined) per year selection and running exploratory statistics test analysis but all simultaneously (histogram) with complex analyzed detector efficiency "properly" offline corrections 40 Eventually 120 PB of Data updated Different format for each DIRAC Grid software Sophisticated specialized Belle II High Monte Carlo and continuously with stage in analysis but data data analysis code Energy Physics observational data sophisticated uniform within each stage followed by basic Experiment selection and exploratory statistics test analysis but all (histogram) with complex analyzed detector efficiency "properly" offline corrections
from successor LSST in 2020s 37 DOE Extreme Data from Cosmological Sky Survey Several petabytes from Dark Energy Survey and Zwicky Transient Factory, simulations > 10 PB Analysis done in batch mode with data from observations and simulations updated daily Image and simulation data MPI, FFTW, viz packages, numpy, Boost, OpenMP, ScaLAPCK, PSQL and MySQL databases, Eigen, cfitsio, astrometry.net, and Minuit2 New analytics needed to analyze simulation results 38 Large Survey Data for Cosmology Petabytes of data from Dark Energy Survey 400 images of 1 GB in size per night Images Linux cluster, Oracle RDBMS server, Postgres PSQL, large memory machines, standard Linux interactive hosts, GPFS; for simulations, HPC resources; standard astrophysics reduction software as well as Perl/Python wrapper scripts Machine learning to find optical transients, Cholesky decomposition for thousands of simulations with matrices of order 1 million on a side and parallel image storage 39 Particle Physics at LHC 15 PB of data (experiment and Monte Carlo combined) per year Data updated continuously with sophisticated selection and test analysis but all analyzed "properly" offline Different format for each stage in analysis but data uniform within each stage Grid-based environment with over 350,000 cores running simultaneously Sophisticated specialized data analysis code followed by basic exploratory statistics (histogram) with complex detector efficiency corrections 40 Belle II High Energy Physics Experiment Eventually 120 PB of Monte Carlo and observational data Data updated continuously with sophisticated selection and test analysis but all analyzed "properly" offline Different format for each stage in analysis but data uniform within each stage DIRAC Grid software Sophisticated specialized data analysis code followed by basic exploratory statistics (histogram) with complex detector efficiency corrections