1. multiple 1. standard 1. high volume -- 1. multiple -- -- streams of real bioinformatics data transfer to security & Light time data to be tools (BLAST, remote batch privacy source stored and HMMER, processing requirements to beamlines analyzed later multiple resource be satisfied 2. sample data to alignment and be analyzed in phylogenetic 35 real time tools, gene callers, sequence feature predictors…), Perl/Python wrapper scripts, Linux Cluster scheduling 1. ~0.1TB per 1. a wide variety -- 1. visualization -- -- -- day at present of the existing mechanisms for Catalina will increase by astronomical data highly Real-Time factor 100 analysis tools, data Transient plus a large parameter spaces Survey amount of custom developed tools and software, some of it a research project in itself 2. automated classification 36 with machine learning tools given the very sparse and heterogeneous data, dynamically evolving in time as more data come in, with follow-up decision making reflecting limited follow up resources 1. ~1 PB per year 1. interpretation 1. MPI, 1. interpretation -- -- -- becoming 7PB a of results from OpenMP, C, of results using DOE year detailed C++, F90, advanced Extreme observational simulations FFTW, viz visualization Data datal requires packages, techniques and advanced python, FFTW, capabilities 37 analysis and numpy, Boost, visualization OpenMP, techniques and ScaLAPCK, capabilities PSQL & MySQL databases, Eigen, cfitsio,
35 Light source beamlines 1. multiple streams of real time data to be stored and analyzed later 2. sample data to be analyzed in real time 1. standard bioinformatics tools (BLAST, HMMER, multiple alignment and phylogenetic tools, gene callers, sequence feature predictors…), Perl/Python wrapper scripts, Linux Cluster scheduling 1. high volume data transfer to remote batch processing resource -- 1. multiple security & privacy requirements to be satisfied -- -- 36 Catalina Real-Time Transient Survey 1. ~0.1TB per day at present will increase by factor 100 1. a wide variety of the existing astronomical data analysis tools, plus a large amount of custom developed tools and software, some of it a research project in itself 2. automated classification with machine learning tools given the very sparse and heterogeneous data, dynamically evolving in time as more data come in, with follow-up decision making reflecting limited follow up resources -- 1. visualization mechanisms for highly data parameter spaces -- -- -- 37 DOE Extreme Data 1. ~1 PB per year becoming 7PB a year observational datal 1. interpretation of results from detailed simulations requires advanced analysis and visualization techniques and capabilities 1. MPI, OpenMP, C, C++, F90, FFTW, viz packages, python, FFTW, numpy, Boost, OpenMP, ScaLAPCK, PSQL & MySQL databases, Eigen, cfitsio, 1. interpretation of results using advanced visualization techniques and capabilities -- -- --