Source: [as of 5.01.2018]. As shown in the table above, metadata management can be grouped in four different categories regarding the of metadata management metadata handling, metadata authority, relationship to – statistical cycle/process and users. For each principle, a common set of indicators may be constructed, depending on the project being implemented. As written in the introduction, metadata may managed in three different ways active, semi-active – and passive. Depending on the metadata store that must be implemented, we can observe that most of the metadata management is performed in passive way, which means that metadata repository is stored in external information system. This allows managing metadata for the whole statistical data and allows better integration of the data. Dealing with metadata management is important to increase the reliability and clearance of the data being processed. Because data quality is a part of the WP8 Quality Report, we will concentrate how to manage and deal with the quality of the metadata. In Table 2. we included metadata quality issues that should be regarded when collecting and processing the information. Table 2. Metadata quality evaluation Quality Dimension Factors to consider Complexity Technical constraints Whether structured or unstructured Readability Presence of hierarchies and nesting Completeness Whether the metadata is available, interpretable and complete Usability Resources required to import and analyse Risk analysis Time-related factors Timeliness Periodicity Changes through time Linkability Presence and quality of linking variables Linking level Coherence - consistency Standardisation Metadata available for key variables (classification variables, construct being measured) Validity Transparency of methods and processes Soundness of methods and processes Source: A Suggested Framework for the Quality of Big Data, Deliverables of the Big Data Quality Task Team, December, 2014, [as of 5.01.2018]. Every factor listed in table above can have a set of indicators. It is important to know that selected factors may not be relevant for various datasets, so the decision of using specific set of indicators may depend on the data source type (e.g., structured or unstructured) and its origin (e.g., administrative data source or web data). Discussion Metadata management is strictly related to data processing. However it is not possible to ensure accurate data management without knowing the quality dimension of the metadata. Having a
Source: [as of 5.01.2018]. As shown in the table above, metadata management can be grouped in four different categories regarding the of metadata management – metadata handling, metadata authority, relationship to statistical cycle/process and users. For each principle, a common set of indicators may be constructed, depending on the project being implemented. As written in the introduction, metadata may managed in three different ways – active, semi-active and passive. Depending on the metadata store that must be implemented, we can observe that most of the metadata management is performed in passive way, which means that metadata repository is stored in external information system. This allows managing metadata for the whole statistical data and allows better integration of the data. Dealing with metadata management is important to increase the reliability and clearance of the data being processed. Because data quality is a part of the WP8 Quality Report, we will concentrate how to manage and deal with the quality of the metadata. In Table 2. we included metadata quality issues that should be regarded when collecting and processing the information. Table 2. Metadata quality evaluation Quality Dimension Factors to consider Complexity Technical constraints Whether structured or unstructured Readability Presence of hierarchies and nesting Completeness Whether the metadata is available, interpretable and complete Usability Resources required to import and analyse Risk analysis Time-related factors Timeliness Periodicity Changes through time Linkability Presence and quality of linking variables Linking level Coherence - consistency Standardisation Metadata available for key variables (classification variables, construct being measured) Validity Transparency of methods and processes Soundness of methods and processes Source: A Suggested Framework for the Quality of Big Data, Deliverables of the Big Data Quality Task Team, December, 2014, [as of 5.01.2018]. Every factor listed in table above can have a set of indicators. It is important to know that selected factors may not be relevant for various datasets, so the decision of using specific set of indicators may depend on the data source type (e.g., structured or unstructured) and its origin (e.g., administrative data source or web data). Discussion Metadata management is strictly related to data processing. However it is not possible to ensure accurate data management without knowing the quality dimension of the metadata. Having a
WP8_Deliverable_8_3_IT_Report_2018_01_09.docx