Watching data management in genomics over the past 15 years has
been a little like witnessing a Neoterics experiment. Each cycle
of data floods, new and incompatible technologies, and
integration issues has resulted, after the initial disruption,
in simpler, faster, and more capable systems. Sample metadata
like species, strain, and provenance were stored in standard lab
notebooks until the first insult—the Human Genome Project.
Lacking teams of software developers or large software budgets,
sequencing labs invented new server data management tools such
as AceDB (enabled by open-source Unix systems). These systems
were more accessible than existing laboratory information
management systems (LIMS) systems, but scientists were still
forced to try to communicate their needs to bioinformaticians,
who had scant understanding of many-to-many relationships and
primary keys. At the same time, client-side solutions like Excel
spreadsheets became commonplace, affording scientists control of
their data, at least until t