Networking Continuous RAW data transfer of ≈20Gbps at designed luminosity between Japan and US Additional transfer rates are currently being investigated Software Open Science Grid, Geant4, DIRAC, FTS, Belle II framework Big Data Data Source Distributed data centers Characteristics (distributed/centralized) Primary data centers are in Japan (KEK) and US (PNNL) Volume (size) Total integrated RAW data ≈120PB and physics data ≈15PB and ≈100PB MC samples Velocity Data will be re-calibrated and analyzed incrementally (e.g. real time) Data rates will increase based on the accelerator luminosity Variety Data will be re-calibrated and distributed incrementally. (multiple datasets, mashup) Variability (rate of Collisions will progressively increase until the designed change) luminosity is reached (3000 BB pairs per sec). Expected event size is ≈300kB per events. Big Data Science Veracity (Robustness Validation will be performed using known reference (collection, curation, Issues) physics processes analysis, Visualization N/A action) Data Quality Output data will be re-calibrated and validated incrementally Data Types Tuple based output Data Analytics Data clustering and classification is an integral part of the computing model. Individual scientists define event level analytics. Big Data Specific Data movement and bookkeeping (file and event level meta-data). Challenges (Gaps) Big Data Specific Network infrastructure required for continuous data transfer between Japan (KEK) and Challenges in Mobility US (PNNL). Security and Privacy No special challenges. Data is accessed using grid authentication. Requirements Highlight issues for generalizing this use case (e.g. for ref. architecture) More Information (URLs) Earth, Environmental and Polar Science> Use Case 41: EISCAT 3D Incoherent Scatter Radar System Use Case Title EISCAT 3D incoherent scatter radar system Vertical (area) Environmental Science Author/Company/Email Yin Chen /Cardiff University/ Ingemar Häggström, Ingrid Mann, Craig Heinselman/ EISCAT Science Association/{
Networking Continuous RAW data transfer of ≈20Gbps at designed luminosity between Japan and US Additional transfer rates are currently being investigated Software Open Science Grid, Geant4, DIRAC, FTS, Belle II framework Big Data Characteristics Data Source (distributed/centralized) Distributed data centers Primary data centers are in Japan (KEK) and US (PNNL) Volume (size) Total integrated RAW data ≈120PB and physics data ≈15PB and ≈100PB MC samples Velocity (e.g. real time) Data will be re-calibrated and analyzed incrementally Data rates will increase based on the accelerator luminosity Variety (multiple datasets, mashup) Data will be re-calibrated and distributed incrementally. Variability (rate of change) Collisions will progressively increase until the designed luminosity is reached (3000 BB pairs per sec). Expected event size is ≈300kB per events. Big Data Science (collection, curation, analysis, action) Veracity (Robustness Issues) Validation will be performed using known reference physics processes Visualization N/A Data Quality Output data will be re-calibrated and validated incrementally Data Types Tuple based output Data Analytics Data clustering and classification is an integral part of the computing model. Individual scientists define event level analytics. Big Data Specific Challenges (Gaps) Data movement and bookkeeping (file and event level meta-data). Big Data Specific Challenges in Mobility Network infrastructure required for continuous data transfer between Japan (KEK) and US (PNNL). Security and Privacy Requirements No special challenges. Data is accessed using grid authentication. Highlight issues for generalizing this use case (e.g. for ref. architecture) More Information (URLs) Earth, Environmental and Polar Science> Use Case 41: EISCAT 3D Incoherent Scatter Radar System Use Case Title EISCAT 3D incoherent scatter radar system Vertical (area) Environmental Science Author/Company/Email Yin Chen /Cardiff University/ Ingemar Häggström, Ingrid Mann, Craig Heinselman/ EISCAT Science Association/{