When it slithers through the grass, the legless glass lizard is indistinguishable from a snake. But harass it and it will perform a very un-snakelike feat. It will leave its tail behind – still wriggling – and slide away. That isn’t the only surprise the glass lizard has in store. A careful look also reveals inflexible jaws, movable eyelids, and ear openings. These are all traits that lizards display but snakes don’t. One way or another, this peculiar creature slithers between the cracks of our familiar categories.

To organise the messy diversity of a million-plus different life forms, we need to sort them into the boxes we call species. And what would be more natural than using visible traits such as legs, jaws or ears for that purpose? About a century before Charles Darwin, the systematist Carl Linnaeus did just that when he created our modern classification of life’s diversity. So did Georges Cuvier, the father of palaeontology, when he classified fossils that had been preserved through the ages.

Classification requires comparison. In the process, we see how deeply similar the legs of birds and lions are, or the flowers of roses and marigolds. Such resemblances form a cornerstone of Darwin’s great insight that all life forms a grand family. Yet scientists such as Cuvier rejected the idea of evolution’s great chain of living beings, drawing support from the large gaps that then existed in the fossil record. ‘If the species have changed by degrees,’ he wrote in 1827, ‘we should find some traces of these gradual modifications.’ If he had seen the intermediate steps that we have now seen, perhaps he would have changed his mind.

But perhaps not. For the reasons to reject evolution go deeper than incomplete knowledge. In fact, we can follow them all the way back to Plato, whose influence looms so large that the 20th-century thinker Alfred North Whitehead could relegate the entirety of European philosophy to a ‘series of footnotes’ to his work.

For Plato, the perceptible material world is like a faint shadow of a higher reality. What really matters is the realm of abstract concepts. To a Platonist, the essence of soccer balls, golf balls and tennis balls is their ball-like shape. It is this pure, abstract and unchanging essence that is real, not the physical balls, whose existence is as fleeting and impermanent as a shadow.

A systematist’s task might be daunting, but it becomes manageable if each species is distinguished by its own Platonic essence. For example, a legless body and flexible jaws might be part of a snake’s essence, different from that of other reptiles. The task is to find a species’ essence. Indeed, the essence really is the species in the world of Platonists. To be a snake is nothing other than to be an instance of the form of the snake.

The only problem: the glass lizard. And hundreds of other creatures that defy easy categorisation, such as Eupodophis, from the late Cretaceous period, a snake with rudimentary hind legs. In an ever-changing Darwinian world, species incessantly spew forth new species whose traits can shade into one another. The 20th-century biologist Ernst Mayr called Plato the ‘great antihero of evolutionism’, and in fact it was Mayr who replaced the essentialist concept of species with a modern biological alternative, based on individuals in the same population that can interbreed.

But as has happened many times before, Plato might have the last word. We just need to look deeper than the ephemeral appearance of living things.

The glass lizard itself comprises billions of cells. Each cell contains thousands of different kinds of proteins – long string-like molecules made of 20 different kinds of amino acid. And each of these proteins has a unique ability. It might catalyse a chemical reaction, or prevent a cell from collapsing, or sense nutrients, or receive signals from other cells, and so on. Each of these abilities was an innovation – a qualitatively new and useful feature that can make the difference between life and death – when it first arose, millions of years ago.

How do random DNA changes lead to innovation? Darwin’s concept of natural selection, although crucial to understand evolution, doesn’t help much. The thing is, selection can only spread innovations that already exist. The botanist Hugo de Vries said it best in 1905: ‘Natural selection can explain the survival of the fittest, but it cannot explain the arrival of the fittest.’ (Half a century earlier, Darwin had already admitted that calling variations random is just another way of admitting that we don’t know their origins.)

A metaphor might help to clarify the problem. Imagine a giant library of books containing all possible sequences of letters in the alphabet. Such a library would be huge beyond imagination, and most of its texts would of course be pure gibberish. But some would contain islands of intelligibility – a word here, a Haiku there – in a sea of random letters. Still others would tell all stories real and imagined: not only Dickens’s Oliver Twist or Goethe’s Faust, but all possible novels and dramas, the biography of every single human, true and false histories of the world, of other worlds as yet unseen, and so on. Some texts would include descriptions of countless technological innovations, from the wheel to the steam engine to the transistor – including countless innovations yet to be imagined. But the chances of choosing such a valuable tome by chance are minuscule.

Evolution can’t look up the chemicals it needs in a giant catalogue. No, it has to inch its way along the stacks

A protein is a volume in a library just like this, written in a 20-letter alphabet of amino acids. And while protein texts might not be as long as Tolstoy’s War and Peace, their total number is still astonishing. For example, a library of every possible amino acid string that is 500 letters long would contain more than 10600 texts – a one with 600 trailing zeros. That vastly outnumbers the atoms in the visible universe.

The library is a giant space of the possible, encoding all the proteins that could be useful to life. But here’s the thing: evolution can’t simply look up the chemicals it needs in a giant catalogue. No, it has to inch its way painstakingly along the stacks. Imagine a crowd of browsers – each one representing an entire familial line – who must blindly explore the library, step by random step. This sounds like a party game, but there’s a grisly twist. A mutation that compromises an essential protein such as haemoglobin is punishable by death. On that ill-fated volume, the bloodline ends.

The challenge, then, is to land on texts that work. Nature has already discovered millions of them. Human engineers have discovered many more, and the pace of discovery shows no signs of slowing. To appreciate the innovative wonders hidden in these libraries, you need to go no further than the bewildering diversity of organisms all around us. Evolution’s giant epic unfolds while its populations scour these libraries.

If you had to find a text on a specific subject in such a library – without a catalogue – you would get utterly lost. Worse than that, if missteps can be fatal, you would quickly die. Yet life not only survived, it found countless new meaningful texts in these libraries. Understanding how it did that requires us to build the catalogue that evolution lacks. It demands that we work out how these libraries are organised to comprehend how innovation through blind search is possible.

For more than a decade, this endeavour has been a focus of my research at the University of Zurich and at the Santa Fe Institute in the US. We evolve molecules in the laboratory and record their journey through these libraries, together with any new and useful texts they find. We also map the locations of millions of molecules that nature’s populations have discovered in their billion-year journey. We use powerful computer simulations to explore those parts of a library that nature has not yet discovered. Through these efforts, we and others have found a system of organisation in these libraries that is as strange as it is perfect for guideless exploration.

One of its features is easily explained once we observe that neighbouring texts in nature’s library have similar letter sequences, and the closest texts – immediate neighbours – differ in just a single letter.

Imagine that you could walk from any meaningful text to a neighbour, and from that to its neighbour, and onward from here, until you had traversed most of the library, altering most letters but leaving the meaning of the text (that is, the function of the protein) intact. Imagine also that you could walk from that first text into a different direction, change a different letter, and another one, and so on, again almost all the way through the library, without changing the text’s meaning. And imagine that you could start this journey in not just one but 100 different ways, each one tracing one of a myriad alternative paths through the library, each encoding only synonymous texts that differ in most letters. Nature’s libraries are just like that, permeated with sprawling networks of synonymous texts – I call them genotype networks – each encoding a molecule and its biochemical function.

If you laid out a human library like this you would be declared mad. It isn’t just that all the books on, say, transistor design would be spread throughout the library. Even more strangely, myriad texts would explain in different ways how to build the same transistor. In normal human libraries we like to have technical manuals in one section, and Darwin’s writings in another, and Tolstoy’s novels in yet another, so that we can make a beeline to whatever grabs our interest. But when you cannot make a beeline, because each step takes you in a random direction prescribed by a DNA mutation, it turns out that sprawling genotype networks are just what you need to survive.

Random DNA changes in some members of a population could disable an essential protein such as haemoglobin and lead to death, but because genotype networks exist, other mutations can create a synonymous text that preserves the protein’s function and saves the organism. This cycle of mutation and natural selection repeats in the survivors’ descendants. Some of them die, but others live and get to take one step further. Step by step, the population of survivors spreads out through the library in a process that unfolds over many generations.

Relatives of the lizard’s oxygen transport protein illustrate how far this exploration can go. They are all descendants of a single long-forgotten ancestral protein that existed more than a billion years ago. By now, they occur not only all over the animal kingdom but even in plants. They have travelled far and wide throughout the library. And still they express the same chemical function. They all bind oxygen.

Just as the word GOLD emerges from a single letter change in MOLD, some neighbours of a text express new meanings

Their amino acid text, however, has diverged beyond recognition. Today’s haemoglobin proteins share as little as four per cent of the letters among their roughly 100 amino acids. It’s as if you could write a poem conveying the same meaning and emotions in myriad different ways. And there are many other molecular poems like it: proteins that help extract energy from nutrients, communicate between our cells, sense the world around us, and so on.

The remarkable thing is, having so many different ways to say the same thing means that there are many more possible slips of the tongue. And with each slip of the tongue comes the possibility of saying something different. Just as the word GOLD emerges from a single letter change in MOLD, some neighbours of a text express new meanings. And as the browsers work their way through each synonym for some original text, different innovations become accessible. By creating safe paths through the library, genotype networks create the very possibility of innovation.

Let me put this point as strongly as I can. Without these pathways of synonymous texts, these sets of genes that express precisely the same function in ever-shifting sequences of letters, it would not be possible to keep finding new innovations via random mutation. Evolution would not work.

So nature’s libraries and their sprawling networks go a long way towards explaining life’s capacity to evolve. But where do they come from? You cannot see them in the glass lizard or its anatomy. They are nowhere near life’s visible surface, nor are they underneath this surface, in the structure of its tissues and cells. They are not even in the submicroscopic structure of its DNA. They exist in a world of concepts, the kind of abstract concepts that mathematicians explore.

Does that make them any less real?

The question whether we create or discover new concepts – especially of the mathematical kind – has occupied humankind for more than 2,500 years, at least since the Pythagoreans, Plato’s intellectual ancestors, declared that ‘all is number’. Some believe with the Austrian philosopher Ludwig Wittgenstein that mathematical truths are human inventions. But others believe with Plato that our visible world is a faint shadow of higher truths. Among them are many mathematicians and physicists, including Charles Fefferman, winner of the Fields medal, the equivalent of a Nobel Prize in mathematics. He expressed his experience when breaking new mathematical ground this way:

There’s something awe-inspiring. You aren’t creating. You’re discovering what was there all the time, and that is much more beautiful than anything that man can create.

In physics, the Nobel laureate Eugene Wigner called it ‘the unreasonable effectiveness of mathematics’. And indeed, it is not clear why Newton’s law of gravitation should apply to so much more than the falling apple that might have inspired it, why it should describe everything from accreting planets to entire solar systems and rotating galaxies. Except that it does. For whatever reason, reality appears to obey certain mathematical formulae.

Nature’s libraries add another dimension to a centuries-old debate about the reality of the Platonic realm. Until now, this debate largely revolved around abstractions like the ones we find in mathematics. With the genotype networks, a new element enters: experimental science.

The law of gravitation is not like the blueprint for a house you can build, but hemoglobin’s text is. We can manufacture this protein – or any other protein we choose from the protein library – and study its chemical meaning with sophisticated instruments. Novel texts found by humans include those encoding heat-resistant enzymes in laundry detergents and insulins that act faster or longer in the body than their natural counterparts. Together with thousands of natural proteins, these inhabitants of the protein library have proven surprising – more than anything we just made up.

Nature’s libraries are the fountains of biological innovation that Darwin was looking for. And unlike the realm of abstract forms that Plato envisioned, they are richer, more diverse, and more complex than the visible world. They harbour enough innovations for all the species Darwinian evolution has created – and could create. No planet would be large enough to explore all of them. The legless lizard and the rest of the living world, in all their glory, are just faint shadows of this Platonic realm of the possible.