Beyond the Quantum review: A remarkable book on quantum mechanics reveals a really big idea

Beyond the Quantum
Antony Valentini, Oxford University Press

Physics, it is fair to say, hasn’t gone to plan. After decades of hopeful searching, dark matter still hasn’t been directly detected. We found the Higgs boson, but nothing to pave the way forward. And string theory, that much-touted theory of everything, has yet to yield a clear, testable prediction. Confidence is low. Where do we go from here?

In recent years, many physicist-authors of popular science have sidestepped that question. Where once they pointed boldly to the next great discovery, they are now often seen retreating into philosophical reflection or re-explaining what we know already. Not so Antony Valentini at Imperial College London. In Beyond the Quantum: A quest for the origin and hidden meaning of quantum mechanics, he presents something almost antiquated in its rarity: a big idea.

As the title suggests, his central target is quantum mechanics, which has underpinned physics for a century. This hinges on the wave function, a mathematical expression that can, so the textbooks say, specify the full state of any system, from a fundamental particle to a cat or even you and me.

The strange thing about the wave function is that it doesn’t usually describe ordinary, localised objects at all, but rather spread-out, fuzzy, wave-like versions of things. Never mind. When we look at an object – again, so the textbooks say – the wave function “collapses” into a familiar yet random outcome, with a probability given by the Born rule (named after physicist Max Born). Only now do we have an object with definite properties in a definite place.

Much as mainstream physics tries to ignore it, the interpretation of the wave function has always been a mystery – and there are essentially only two realistic answers. One is that the wave function really does describe reality – that electrons, cats and people genuinely exist in many states at once, spread out across space and possibility. This is the many-worlds interpretation, with its vast metaphysical implications.

The other answer is that the wave function isn’t the whole story. The dominant theory here, much developed by Valentini, is pilot-wave theory, first proposed by theorist Louis de Broglie in 1927 and later revived by physicist David Bohm.

Pilot-wave theory takes the wave function to be real but incomplete. It suggests that it acts as a guiding structure for individual particles, rather like waves steering plastic bottles floating on the sea. The particles themselves are never spread out or indeterminate – their wave-like behaviour comes from the pilot wave and where they sit on it.

Pilot-wave theory has long been known to reproduce all the predictions of quantum mechanics, with no fundamental randomness. But, as Valentini stresses, this agreement rests on an assumption: that particles are in equilibrium with the wave, distributed in just the right way. That assumption fits today’s experimental data – the results are practically undeniable – but it need not always have held.

Valentini proposes that, in the early universe, particles were distributed far from quantum equilibrium, before “relaxing” into their present state, rather as a cup of coffee cools to match its surroundings. In this view, the Born rule and its randomness are no longer basic features of nature, but historical accidents – they are by-products of cosmology.

That striking payoff isn’t the only one. Quantum randomness also prevents any practical exploitations of non-locality, the immediate interaction of objects separated by time and space. If the Born rule didn’t hold in the early universe, Valentini argues, instantaneous communication across vast distances would have been possible, perhaps leaving subtle imprints in the cosmic microwave background. If any relics from this era still exist, such superluminal signalling might be achievable now.

Given the lack of evidence, this might sound a bit wild were it not for Valentini’s meticulous analysis of how orthodox quantum mechanics came to hold sway (the book is worth reading for this alone). If there is one weakness, it is the lack of a plain description of the pilot wave. Still, whether or not it proves to be true, Valentini’s work at least shows us that, in a field short on confidence, this is what a genuinely big idea looks like.

Jon Cartwright is a writer based in Bristol, UK