sustainable and extendable) DQA procedure well Level Level 4 + Data Quality Data quality documented, fully Assurance Ad Hoc and random DQA procedure monitored and assurance implemented and available DQA procedure defined reported (DQA) DQA procedure not online with master reference and documented and procedure (The state of data defined and Conforming to community quality data partially implemented unknown or quality being documented metadata & standards Limited data quality none assured) External review assurance metadata Level 2+ Level Level 3 + Level 4 + None or Data Quality Sampling and Sampling and analysis are Anomaly detection Cross-validation of temporal & Control/Monitoring analysis are regular frequent and systematic but Sampling procedure well-documented spatial characteristics in time and space not automatic unknown or and fully implemented using Physical consistency check spotty community metrics, (The state of data Limited Community metrics Conforming to community quality automatic, tracked and quality being metrics defined and partially Analysis metadata & standards reported defined & implemented controlled and unknown or implemented Dynamic providers/users feedback monitored) random in time Limited quality monitoring Procedure documented and in place metadata available online Level Level 4 + Data Quality Level 3 + Assessment Level 1 + Level 2 + Assessment performed on a recurring basis theoretical Quality metadata assessed Research product Operational product basis assessed (The state of data assessed (methods assessed (methods and Conforming to community quality Limited quality assessment (methods and quality being and results online) results online) metadata & standards metadata results online) assessed) External ranking Algorithm Theoretical Level Basis Document (ATBD) & source code online Transparency Dataset configuration Limited /Traceability managed (CM) Level 3 + product Level 4 + Product information Unique Object Identifier Operational Algorithm information (The state of being available in System information online (OID) assigned (dataset, Description (OAD) online, available literature transparent, documentation, source OID assigned, and under Complete data provenance online code) CM trackable, and traceable) Data citation tracked (e.g., utilizing Digital Object Identifier (DOI) system) Level 4 + Level Data Integrity Data ingest integrity Level 2 + Level 3 + Unknown or Data authenticity verifiable verifiable no data ingest (e.g., data signature technology) (The state of data Data archive integrity Data access integrity (e.g., checksum integrity check integrity being verifiable verifiable Performance of data integrity check technology) verifiable) monitored and reported
sustainable and extendable) Data Quality Assurance (The state of data quality being assured) Data quality assurance (DQA) procedure unknown or none Ad Hoc and random DQA procedure not defined and documented DQA procedure defined and documented and partially implemented DQA procedure well documented, fully implemented and available online with master reference data Limited data quality assurance metadata Level 4 + DQA procedure monitored and reported Conforming to community quality metadata & standards External review Level Data Quality Control/Monitoring (The state of data quality being controlled and monitored) None or Sampling unknown or spotty Analysis unknown or random in time Sampling and analysis are regular in time and space Limited metrics defined & implemented Level 2+ Sampling and analysis are frequent and systematic but not automatic Community metrics defined and partially implemented Procedure documented and available online Level 3 + Anomaly detection procedure well-documented and fully implemented using community metrics, automatic, tracked and reported Limited quality monitoring metadata Level 4 + Cross-validation of temporal & spatial characteristics Physical consistency check Conforming to community quality metadata & standards Dynamic providers/users feedback in place Level Data Quality Assessment (The state of data quality being assessed) theoretical basis assessed (methods and results online) Level 1 + Research product assessed (methods and results online) Level 2 + Operational product assessed (methods and results online) Level 3 + Quality metadata assessed Limited quality assessment metadata Level 4 + Assessment performed on a recurring basis Conforming to community quality metadata & standards External ranking Level Transparency /Traceability (The state of being transparent, trackable, and traceable) Limited product information available Product information available in literature Algorithm Theoretical Basis Document (ATBD) & source code online Dataset configuration managed (CM) Unique Object Identifier (OID) assigned (dataset, documentation, source code) Data citation tracked (e.g., utilizing Digital Object Identifier (DOI) system) Level 3 + Operational Algorithm Description (OAD) online, OID assigned, and under CM Level 4 + System information online Complete data provenance online Level Data Integrity (The state of data integrity being verifiable) Unknown or no data ingest integrity check Data ingest integrity verifiable (e.g., checksum technology) Level 2 + Data archive integrity verifiable Level 3 + Data access integrity verifiable Level 4 + Data authenticity verifiable (e.g., data signature technology) Performance of data integrity check monitored and reported Level