Minuit2 38 Petabytes of data 400 images of 1 GB Images Linux cluster, Oracle Machine learning to find Large Survey Data from Dark Energy in size per night RDBMS server, Postgres optical transients, for Cosmology Survey PSQL, large memory Cholesky decomposition machines, standard for thousands of Linux interactive hosts, simulations with matrices GPFS; for simulations, of order 1 million on a side HPC resources; standard and parallel image storage astrophysics reduction software as well as Perl/Python wrapper scripts 39 15 PB of data Data updated Different format for each Grid-based environment Sophisticated specialized Particle Physics at (experiment and continuously with stage in analysis but data with over 350,000 cores data analysis code LHC Monte Carlo sophisticated uniform within each stage running simultaneously followed by basic combined) per year selection and exploratory statistics test analysis but all (histogram) with complex analyzed detector efficiency "properly" offline corrections 40 Eventually 120 PB of Data updated Different format for each DIRAC Grid software Sophisticated specialized Belle II High Monte Carlo and continuously with stage in analysis but data data analysis code Energy Physics observational data sophisticated uniform within each stage followed by basic Experiment selection and exploratory statistics test analysis but all (histogram) with complex analyzed detector efficiency "properly" offline corrections 41 Terabytes/year Data updated Big data uniform Custom analysis based Pattern recognition, EISCAT 3D (current), 40 PB/year continuously with on flat file data storage demanding correlation incoherent starting ~2022 real-time test routines, high-level scatter radar analysis and batch parameter extraction system full analysis 42 Low volume (apart Mainly real-time Six separate projects with R and Python Data assimilation, ENVRI from EISCAT 3D data streams common architecture for (Matplotlib) for (statistical) analysis, data Environmental given above), one infrastructure, data very visualization, custom mining, data extraction, Research system EPOS ~15 diverse across projects software for processing scientific modeling and Infrastructure TB/year simulation, scientific
Minuit2 38 Large Survey Data for Cosmology Petabytes of data from Dark Energy Survey 400 images of 1 GB in size per night Images Linux cluster, Oracle RDBMS server, Postgres PSQL, large memory machines, standard Linux interactive hosts, GPFS; for simulations, HPC resources; standard astrophysics reduction software as well as Perl/Python wrapper scripts Machine learning to find optical transients, Cholesky decomposition for thousands of simulations with matrices of order 1 million on a side and parallel image storage 39 Particle Physics at LHC 15 PB of data (experiment and Monte Carlo combined) per year Data updated continuously with sophisticated selection and test analysis but all analyzed "properly" offline Different format for each stage in analysis but data uniform within each stage Grid-based environment with over 350,000 cores running simultaneously Sophisticated specialized data analysis code followed by basic exploratory statistics (histogram) with complex detector efficiency corrections 40 Belle II High Energy Physics Experiment Eventually 120 PB of Monte Carlo and observational data Data updated continuously with sophisticated selection and test analysis but all analyzed "properly" offline Different format for each stage in analysis but data uniform within each stage DIRAC Grid software Sophisticated specialized data analysis code followed by basic exploratory statistics (histogram) with complex detector efficiency corrections 41 EISCAT 3D incoherent scatter radar system Terabytes/year (current), 40 PB/year starting ~2022 Data updated continuously with real-time test analysis and batch full analysis Big data uniform Custom analysis based on flat file data storage Pattern recognition, demanding correlation routines, high-level parameter extraction 42 ENVRI Environmental Research Infrastructure Low volume (apart from EISCAT 3D given above), one system EPOS ~15 TB/year Mainly real-time data streams Six separate projects with common architecture for infrastructure, data very diverse across projects R and Python (Matplotlib) for visualization, custom software for processing Data assimilation, (statistical) analysis, data mining, data extraction, scientific modeling and simulation, scientific