Variety Measurements: different versions, formats, replicas, (multiple datasets, external sources ... mashup) System information: configuration, monitoring, logs/provenance ... Users’ metadata/data: experiments, analysis, sharing, communications … Variability (rate of In time, instantly, a few ms. change) Along the radar beams, 100ns. Running 24/7, EISCAT_3D have very high demands Big Data Science Veracity (Robustness on robustness. (collection, curation, Issues) Data and performance assurance is vital for the analysis, and archive systems. These systems must be action) able to guarantee to meet minimum data rate acceptance at all times or scientific data will be lost. Similarly the systems must guarantee that data held is not volatile or corrupt. This latter requirement is particularly vital at the permanent archive where data is most likely to be accessed by scientific users and least easy to check; data corruption here has a significant possibility of being non-recoverable and of poisoning the scientific literature. Real-time visualisation of analysed data, e.g., Visualization with a figure of updating panels showing electron density, temperatures and ion velocity to those data for each beam. Non-real-time (post-experiment) visualisation of the physical parameters of interest, e.g., by standard plots, using three-dimensional block to show to spatial variation (in the user selected cuts), using animations to show the temporal variation, allow the visualisation of 5 or higher data, e.g., using the 'cut up and stack' technique to reduce the dimensionality, that is take one or more independent coordinates as discrete; or volume rendering technique to display a 2D projection of a 3D discretely sampled data set. (Interactive) Visualisation. E.g., to allow users to combine the information on several spectral features, e.g., by using colour coding, and to provide real-time visualisation facility to allow the users to link or plug in tailor-made data visualisation functions, and more importantly functions to signal for special observational conditions. Data Quality Monitoring software will be provided which allows The Operator to see incoming data via the
Variety (multiple datasets, mashup) Measurements: different versions, formats, replicas, external sources ... System information: configuration, monitoring, logs/provenance ... Users’ metadata/data: experiments, analysis, sharing, communications … Variability (rate of change) In time, instantly, a few ms. Along the radar beams, 100ns. Big Data Science (collection, curation, analysis, action) Veracity (Robustness Issues) Running 24/7, EISCAT_3D have very high demands on robustness. Data and performance assurance is vital for the and archive systems. These systems must be able to guarantee to meet minimum data rate acceptance at all times or scientific data will be lost. Similarly the systems must guarantee that data held is not volatile or corrupt. This latter requirement is particularly vital at the permanent archive where data is most likely to be accessed by scientific users and least easy to check; data corruption here has a significant possibility of being non-recoverable and of poisoning the scientific literature. Visualization Real-time visualisation of analysed data, e.g., with a figure of updating panels showing electron density, temperatures and ion velocity to those data for each beam. Non-real-time (post-experiment) visualisation of the physical parameters of interest, e.g., by standard plots, using three-dimensional block to show to spatial variation (in the user selected cuts), using animations to show the temporal variation, allow the visualisation of 5 or higher data, e.g., using the 'cut up and stack' technique to reduce the dimensionality, that is take one or more independent coordinates as discrete; or volume rendering technique to display a 2D projection of a 3D discretely sampled data set. (Interactive) Visualisation. E.g., to allow users to combine the information on several spectral features, e.g., by using colour coding, and to provide real-time visualisation facility to allow the users to link or plug in tailor-made data visualisation functions, and more importantly functions to signal for special observational conditions. Data Quality Monitoring software will be provided which allows The Operator to see incoming data via the