The Search for What Lies Beyond the Quantum

An excerpt from the Epilogue of Einstein’s Unfinished Revolution
by Lee Smolin

A book project is a kind of mental therapy, which forces you to examine your confused thoughts and intuitions and develop them to their logical conclusions. So now that I have written a book which argues that a radically new theory is needed to solve the foundational issues in physics and cosmology, what am I going to do about it? Do I keep to the same safe, hedged program, or go all out on an attempt to solve the real problems?

To bet that the truth requires something as yet undiscovered, we must spend our time searching for that unknown completion. We can’t just sail down one shoreline and up another. We head west: out of sight of land, following our own compass, or the best facsimile thereof that we can cobble together from the clues we take seriously.

There is no more reasonable bet than that our current knowledge is incomplete. In every era of the past our knowledge was incomplete; why should our period be any different? Certainly the puzzles we face are at least as formidable as any in the past. But almost no one bets this way. This puzzles me.

I suspect it’s hard for many physicists to imagine that we are not near the end of our search for the ultimate laws of nature. We have been raised in a culture in which it’s all about having the right answer, and we owe our careers to having been the scientists who had them. But I’ve always had in my head an image of how much more people in the future will know, and how silly our claims to knowledge will look to them. This has probably made me a less effective advocate of my own ideas.

So what do we do with the partly successful inventions, such as loop quantum gravity? At first, the discovery of a new possible direction, incomplete and without experimental confirmation (in other words, highly vulnerable to criticism, as most new theories are at birth), is very worth our time and focus. That X, however incompletely formulated, is something that just might be true, or be part of the truth, even without positive evidence, is certainly good for a decade of examination. But after a third or more of a century, during which many career-long efforts have failed to budge might be true any closer to must be true, isn’t it time to move on? You might think I’m repeating polemics from the string wars, but I’m thinking, with a great deal of affection, of all of us whose years of hard work have failed to yield the breakthroughs we fantasized about. Including myself; especially myself.

Why do we write more and more papers on approaches whose deficiencies have been obvious for decades, and almost no papers proposing new completions of quantum mechanics? It is not for lack of caring, for everyone I know who works on quantum foundations has chosen that risky path because they care passionately about how nature resolves the measurement problem and the other puzzles.

I, for one, am tired of arguing over the ins and outs and relative merits of the existing approaches, and the clever fixes invented to save an idea that is pretty obviously collapsing from insufficiency. So I have a decision to make: I either keep on the present path, which will end up on the top of that low hill just past the next village, or head down into the swamps to stumble along unknown paths in search of undiscovered mountains. If I take the swamp trail, I will almost certainly fail, but I hope to send back reports to interest and inspire those few others who feel in their bones the cost of our ignorance, of giving up the search too soon.

Even if I’m convinced that something very new is needed, I have little idea how to search for scientific truth except by building on an existing research program, using a well-honed tool kit and methodology. This is research as it is taught, recognized, funded, and rewarded by the academic community. A community, I should mention, that it is necessary to be an active part of to get your work taken seriously by people who know enough to evaluate it. What would I put in my research proposals, if my ideas are not expressible in the language of an already existing and widely followed research program? What problems do I set for my PhD students, if they are not to calculate something using tools developed within a given framework? Do I tell my students to wake up in the morning, make coffee, open a blank notebook, and stare at it until a disheveled angel arrives with a revelation? Is that what I should do myself? How many days, weeks, months, years, how many incoherent scribbled pages, do I tolerate before giving up?

It is not just that to try to invent a whole new physics is risky for my career and damaging to my emotional stability. I don’t even know how to begin. Almost no one alive has done that, in the way that the revolutionaries of a century ago did. In my experience there is little as terrifying as putting aside the basic principles that form the foundation of our understanding of how we fit into nature—isn’t that why it feels comforting to know them?

It certainly is easier to work within an existing framework, to test the limits of what we know from the inside, so to speak. We can do this while keeping an open mind about the basic principles and looking out for opportunities to modify those principles or even introduce new ones. Even more important is to keep on the lookout for new opportunities to test theories against experiments and observations. This is what I have done for most of my career, and I venture to say this is true as well of many who work on the main approaches, such as string theory and loop quantum gravity. What we have to show for this is a collection of beautiful results, which may or may not lead to the true story, and, especially precious, a few proposals for new principles, including the holographic principle and the principle of relative locality. But, with all due respect to those of us who invested most of our time in reasonable approaches to the development of reasonable theories, it does not seem to have been enough, this time.

I say to myself, I’ll take such risks after I get my PhD, after I get my postdoc, after my faculty position, after tenure. Bet even tenured, senior, famous professors must apply for research grants, and there is always that fancy career-culminating prize or that comfortable and prestigious chair. So we’ll just wait till retirement. Then we’ll be free to take the big risks. Well, as someone closing in on that, I can report that the only thing you learn is certain, as your fifties and sixties rush by, every day busy with a schedule full of seminars, faculty meetings, working with students, classes, review panels, airplanes, hotels, and conference talks, is that you are not immortal. So maybe it’s all up to a brilliant student somewhere, impossibly arrogant, as the young Einstein was, but blindingly talented enough to absorb the essentials of all we have done, before putting them to one side and confidently starting over.

A friend once told me that the academic world was modeled on monasteries, which were designed to perpetuate old knowledge while resisting the new. Even after decades in the system I am amazed at how the fine mechanics of this work. There is no arguing with the logic of academic fame, which rewards every scientific success with distractions that make it harder to do more science, while imposing enormous disincentives to putting aside polishing your legacy to take on new challenges.

The academic world is very well suited to support what Thomas Kuhn called normal science. That is great until it becomes long past due to complete a revolution.

To my knowledge, few have stumbled on a major discovery by accident; most true breakthroughs were found after years and years of hard, unrewarding work. Feynman said to discover something new you have to take the time to make every mistake possible along the way. And he surely knew.

So I have no better answer than to face the blank notebook. We do have role models. Einstein did it. Bohr did it. De Broglie, Schrodinger, and Heisenberg did it, as did Bohm and Bell. They each found a path from that blank page to a foundational discovery that enlarged our understanding of how nature works. Start by writing down what you are confident we know for sure. Ask yourself which of the fundamental principles of the present canon must survive the coming revolution. That’s the first page. Then turn again to a blank page and start thinking.


Lee Smolin is a theoretical physicist who is a founding and senior faculty member at Perimeter Institute for Theoretical Physics. His previous books include Life of the Cosmos, The Singular Universe and the Reality of Time, Time Reborn, The Trouble with Physics, and Three Roads to Quantum Gravity.





Click here to purchase Einstein’s Unfinished Revolution






*All of the excerpts on my website are from books that have stayed with me for some reason—because the concept was awe inspiring, changed how I view the world, was beautifully expressed, or all three. ‎I personally curate all of the book excerpts and always obtain the author’s approval before posting their work.

Photo Credit: Mibby23 (See image on Flickr HERE)