As the data propagates through the ecosystem, it is being processed and transformed in different ways in order to extract the value from the information. Transformation sub-components can be implemented by independent stakeholders and deployed as stand-alone services. Each transformation function can use different specialized data infrastructure or capabilities best for its requirements, and can have its own privacy and other policy considerations. In its role, Transformation Provider typically executes the manipulations of the data lifecycle of a specific vertical system to meet the requirements or instructions established by the System Orchestrator. [Editor’s Note: Listed activities need to be aligned with the sub-components shown on the diagram.] Transformation Provider activities include: Data Collecti (connect, transport, stage): obtains connection to Data Provider APIs to collect into local system, or to access dynamically when requested. At the initial collection stage, sets of data (e.g., data records) of similar structure are collected (and combined) resulting in uniform security considerations, policies, etc. Initial metadata is created (e.g., subjects with keys are identified) to facilitate subsequent aggregation or lookup method(s). Data Curati: provides cleansing, outlier removal, standardization for the ingestion and storage processes. Data Aggregati: Data from different data providers with easily correlated metadata (e.g., identical keys) is aggregated into a single data set. As a result, the information about each object is enriched or the number of objects in the collection grows. Security considerations and policies concerning the resultant data set are typically similar to the original data. Data Matching: Data from different data providers with disparate metadata (e.g., keys) are aggregated into a single data set. As a result, the information about each object is enriched. The security considerations and policies concerning the resultant data set may differ from the original policies. Data Optimizi (Pre-analytics): determines the appropriate data manipulations and indexes to optimize subsequent transformation processes. Data Analysis: implements the techniques to extract knowledge from the data based on the requirements of the data scientist, who has specified the algorithms to process the data to produce new insights that will address the technical goal. Data Transfer: facilitates secure transfer of data between different repositories and/or between the Transformation and the Capabilities RA blocks. System Service Abstraction TBD
As the data propagates through the ecosystem, it is being processed and transformed in different ways in order to extract the value from the information. Transformation sub-components can be implemented by independent stakeholders and deployed as stand-alone services. Each transformation function can use different specialized data infrastructure or capabilities best for its requirements, and can have its own privacy and other policy considerations. In its role, Transformation Provider typically executes the manipulations of the data lifecycle of a specific vertical system to meet the requirements or instructions established by the System Orchestrator. [Editor’s Note: Listed activities need to be aligned with the sub-components shown on the diagram.] Transformation Provider activities include: Data Collecti (connect, transport, stage): obtains connection to Data Provider APIs to collect into local system, or to access dynamically when requested. At the initial collection stage, sets of data (e.g., data records) of similar structure are collected (and combined) resulting in uniform security considerations, policies, etc. Initial metadata is created (e.g., subjects with keys are identified) to facilitate subsequent aggregation or lookup method(s). Data Curati: provides cleansing, outlier removal, standardization for the ingestion and storage processes. Data Aggregati: Data from different data providers with easily correlated metadata (e.g., identical keys) is aggregated into a single data set. As a result, the information about each object is enriched or the number of objects in the collection grows. Security considerations and policies concerning the resultant data set are typically similar to the original data. Data Matching: Data from different data providers with disparate metadata (e.g., keys) are aggregated into a single data set. As a result, the information about each object is enriched. The security considerations and policies concerning the resultant data set may differ from the original policies. Data Optimizi (Pre-analytics): determines the appropriate data manipulations and indexes to optimize subsequent transformation processes. Data Analysis: implements the techniques to extract knowledge from the data based on the requirements of the data scientist, who has specified the algorithms to process the data to produce new insights that will address the technical goal. Data Transfer: facilitates secure transfer of data between different repositories and/or between the Transformation and the Capabilities RA blocks. System Service Abstraction TBD