Auditable and reusable crosswalks for fast, scaled integration of scattered tabular data
<|reference_start|>Auditable and reusable crosswalks for fast, scaled integration of scattered tabular data: This paper presents an open-source curatorial toolkit intended to produce well-structured and interoperable data. Curation is divided into discrete components, with a schema-centric focus for auditable restructuring of complex and scattered tabular data to conform to a destination schema. Task separation allows development of software and analysis without source data being present. Transformations are captured as high-level sequential scripts describing schema-to-schema mappings, reducing complexity and resource requirements. Ultimately, data are transformed, but the objective is that any data meeting a schema definition can be restructured using a crosswalk. The toolkit is available both as a Python package, and as a 'no-code' visual web application. A visual example is presented, derived from a longitudinal study where scattered source data from hundreds of local councils are integrated into a single database.<|reference_end|>
@article{chait2024auditable,
    title={Auditable and reusable crosswalks for fast, scaled integration of
  scattered tabular data},
    author={Gavin Chait},
    journal={Proceedings of the EuSpRIG 2024 Conference "Spreadsheet
  Productivity & Risks" ISBN : 978-1-905404-59-9},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.01517},
    primaryClass={cs.DB}
}