Quantum Reality Read online

  Quantum Reality

  Great Clarendon Street, Oxford, ox2 6dp, United Kingdom

  Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries

  © Jim Baggott 2020

  The moral rights of the author have been asserted

  First Edition published in 2020

  Impression: 1

  All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by licence or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above

  You must not circulate this work in any other form and you must impose this same condition on any acquirer

  Published in the United States of America by Oxford University Press

  198 Madison Avenue, New York, NY 10016, United States of America

  British Library Cataloguing in Publication Data

  Data available

  Library of Congress Control Number: 9780198830160

  ISBN 978–0–19–883015–3

  ebook ISBN 978–0–19–256583–9

  Printed and bound in Great Britain by Clays Ltd, Elcograf S.p.A.

  To Ian Mills

  Who taught me quite a lot about quantum mechanics

  Contents

  About the Author

  Preamble

  Prologue: Why Didn’t Somebody Tell Me About All This Before?

  Part I The Rules of the Game

  1 The Complete Guide to Quantum Mechanics (Abridged)

  Everything You’ve Ever Wanted to Know, and a Few Things You Didn’t

  2 Just What is This Thing Called ‘Reality’, Anyway?

  The Philosopher and the Scientist: Metaphysical Preconceptions and Empirical Data

  3 Sailing on the Sea of Representation

  How Scientific Theories Work (and Sometimes Don’t)

  4 When Einstein Came Down to Breakfast

  Because You Can’t Write a Book About Quantum Mechanics without a Chapter on the Bohr–Einstein Debate

  Part II Playing the Game

  5 Quantum Mechanics is Complete So Just Shut Up and Calculate

  The View from Scylla: The Legacy of Copenhagen, Relational Quantum Mechanics, and the Role of Information

  6 Quantum Mechanics is Complete But We Need to Reinterpret What it Says

  Revisiting Quantum Probability: Reasonable Axioms, Consistent Histories, and QBism

  7 Quantum Mechanics is Incomplete So We Need to Add Some Things

  Statistical Interpretations Based on Local and Crypto Non-local Hidden Variables

  8 Quantum Mechanics is Incomplete So We Need to Add Some Other Things

  Pilot Waves, Quantum Potentials, and Physical Collapse Mechanisms

  9 Quantum Mechanics is Incomplete Because We Need to Include My Mind (or Should That be Your Mind?)

  Von Neumann’s Ego, Wigner’s Friend, the Participatory Universe, and the Quantum Ghost in the Machine

  10 Quantum Mechanics is Incomplete Because…. Okay, I Give Up

  The View from Charybdis: Everett, Many Worlds, and the Multiverse

  Epilogue: I’ve Got a Very Bad Feeling about This

  Appendix: Realist Propositions and the Axioms of Quantum Mechanics

  Acknowledgements

  List of Figure Acknowledgements

  Endnotes

  Bibliography

  Index

  About the Author

  Jim Baggott is an award-winning science writer. A former academic scientist, he now works as an independent business consultant but maintains a broad interest in science, philosophy, and history and continues to write on these subjects in his spare time. His previous books have been widely acclaimed and include the following:

  The Quantum Cookbook: Mathematical Recipes for the Foundations of Quantum Mechanics

  Quantum Space: Loop Quantum Gravity and the Search for the Structure of Space, Time, and the Universe

  Mass: The Quest to Understand Matter from Greek Atoms to Quantum Fields

  Origins: The Scientific Story of Creation

  Farewell to Reality: How Fairy-tale Physics Betrays the Search for Scientific Truth

  Higgs: The Invention and Discovery of the ‘God Particle’

  The Quantum Story: A History in 40 Moments

  Atomic: The First War of Physics and the Secret History of the Atom Bomb 1939–49, short-listed for the Duke of Westminster Medal for Military Literature

  A Beginner’s Guide to Reality

  Beyond Measure: Modern Physics, Philosophy, and the Meaning of Quantum Theory

  Perfect Symmetry: The Accidental Discovery of Buckminsterfullerene

  The Meaning of Quantum Theory: A Guide for Students of Chemistry and Physics

  Preamble

  I know why you’re here.

  You know that quantum mechanics is an extraordinarily successful scientific theory, on which much of our modern, tech-obsessed lifestyles depend, from smartphones to streaming to satellites. You also know that it is completely mad. Its discovery forced open the window on all those comfortable notions we had gathered about physical reality from our naïve interpretation of Isaac Newton’s laws of motion, and unceremoniously shoved them out. Although quantum mechanics quite obviously works, it appears to leave us chasing ghosts and phantoms, particles that are waves and waves that are particles, cats that are at once both alive and dead, lots of seemingly spooky goings-on, and a desperate desire to lie down quietly in a darkened room.

  But, hold on. If we’re prepared to be a little more specific about what we mean when we talk about ‘reality’ and a little more circumspect about how we think a scientific theory might represent such a reality, then all the mystery goes away.

  I’m not kidding. I have a bit of a reputation as the kind of guy you might find in the kitchen at parties; the kind who spoils all the fun, bursting the bubbles of excitable mystery and urban myth (what Americans sometimes call ‘woo’) with a cold scepticism and a calculating rationality. Spock, not Kirk (or McCoy). One commentator recently called me ‘depressingly sane’.* This is a badge I’m happy to wear with pride. There are many popular books you can buy about the weirdness and the ‘woo’ of quantum mechanics. This isn’t one of them.

  And in any case that’s not why you’re here.

  But—let’s be absolutely clear—a book that says, ‘Honestly, there is no mystery’ would not only be a bit dull and uninteresting (no matter how well it was written), it would also be completely untrue. For sure we can rid ourselves of all the mystery in quantum mechanics but only by abandoning any hope of deepening our understanding of nature. We must become content to use the quantum representation simply as a way to perform calculations and make predictions, and we must resist the temptation to ask: But how does nature actually do that? And there lies the rub: for what is the purpose of a scientific theory if not to aid our understanding of the physical world?

  Let’s be under no illusions. The choice we face is a philosophical one. There is absolutely nothing scientifically wrong with a depressingly sane interpretation of quantum mechanics in which there is no mystery. If we choose instead to pull on the loose thread we are inevitably obliged to take the quantum representation at face value, and interpret its concepts rather more literally. Surprise, surprise. The fabric unravels to give us all those things about the quantum world
that we find utterly baffling, and we’re right back where we started.

  My purpose in this book is (hopefully) not to spoil your fun, but to try to explain what it is about quantum mechanics that forces us to confront this kind of choice, and why this is entirely philosophical in nature. Making different choices leads to different interpretations or even modifications of the quantum representation and its concepts, in what I call (with acknowledgements to George R. R. Martin) the game of theories.

  Part I opens with a brief summary of everything you might need to know about quantum mechanics, which should hopefully set you up for what follows. I will then tell you about the rules of the game, based on a pragmatic but perfectly reasonable understanding of what we mean by ‘reality’, and the kinds of things we can hope to learn from a scientific representation of this. Part I concludes with Albert Einstein’s great debate with Niels Bohr in the late 1920s and early 1930s, and the emergence of the anti-realist Copenhagen interpretation, which admirably sets the scene.

  We will then go on in Part II to look at various attempts to play the game, from the legacy of Copenhagen, through relational quantum mechanics, to interpretations based on quantum ‘information’. We will look at attempts to redefine quantum probability, by reformulating the axioms of quantum mechanics, introducing the notion of consistent histories, and quantum Bayesianism. We then turn our attention to realist interpretations based on the idea that quantum mechanics is a statistical theory. These include hidden variable theories of local (Bell’s inequality), and ‘crypto’ non-local (Leggett’s inequality) varieties.

  Experimental evidence gathered over the past forty years or so comes down pretty firmly against local and crypto non-local hidden variables. So we turn to interpretations based on non-local hidden variables (such as so-called ‘pilot wave’ theories) or we try to fix problems associated with the ‘collapse of the wavefunction’ by introducing new physical mechanisms, including a possible role for human consciousness. We conclude with the notion that the wavefunction is real but doesn’t collapse, which leads to many worlds and the multiverse.

  If you will indulge me, through all of this I will make use of a no doubt overfanciful analogy or metaphor.1 This is based on the notion that the game of theories involves navigating the ‘Ship of Science’ on the perilous ‘Sea of Representation’. Yes, I’ve obviously read too many fantasy novels.

  We sail the ship back and forth between two shores. These are the deceptively welcoming, soft, sandy beaches of Metaphysical Reality and the broken, rocky, and often inhospitable shores of Empirical Reality. The former are shaped by our abstract imaginings, free-wheeling creativity, personal values and prejudices, and a variety of sometimes pretty mundane things we’re obliged to accept without proof in order to do any kind of science at all. These become translated into one or more metaphysical preconceptions, which summarize how we think or even come to believe reality should be. These are beliefs that, by their nature, are not supported by empirical evidence. So, if you prefer you could think of these preconceptions as intuitions or even articles of faith, echoing one of my favourite Einstein quotes: ‘I have no better expression than the term “religious” for this trust in the rational character of reality and in its being accessible, to some extent, to human reason.’2

  Within the sea I have charted two grave dangers. The rock shoal of Scylla lies close to the shores of Empirical Reality. It is a rather empty instrumentalism, perfectly empirically adequate but devoid of any real physical insight and understanding. Charybdis lies close to the beaches of Metaphysical Reality. It is a whirlpool of wild, unconstrained metaphysical nonsense. The challenge to theorists is to discover safe passage across the Sea of Representation. In Quantum Reality I want to explain why this has proven so darn difficult, and why I have a very bad feeling about it.

  So, welcome. You’re here because you want some answers. Please take a seat and make yourself comfortable, and I’ll go and put the kettle on.

  * This was theoretical physicist Sabine Hossenfelder, referencing my book Farewell to Reality: How Fairy-tale Physics Betrays the Search for Scientific Truth, in a tweet dated 11 March 2018.

  Prologue

  Why Didn’t Somebody Tell Me About All This Before?

  My first encounter with quantum mechanics occurred in my very first term as an undergraduate, studying for a bachelor’s degree in chemistry in a rather damp and gloomy Manchester, England, in the autumn of 1975.

  Looking back, it’s no real surprise that all the students in my class (me included) were utterly baffled by what we were taught. Until that moment, we had all been blissfully unaware that there was anything more to be learned about the physical world beyond the smooth continuity and merciless certainties of Newton’s clockwork mechanics.

  Our understanding of atoms was limited to the ‘planetary model’ associated with the names of physicists Niels Bohr and Ernest Rutherford. If we had thought about it at all (and I can tell you that we really hadn’t), then we would have supposed that the classical theories we use to describe planets orbiting the Sun could simply be extended to describe little balls of electrically charged matter orbiting the central nucleus of an atom. Yes, the forces are different, but surely the results would be much the same.

  But now we were told that the physics of atoms and molecules is governed by a very different set of laws, with which even chemists must come to terms. Nothing had prepared us for this. In our first lecture we chomped our way through Max Planck’s discovery of the quantum, Einstein’s ‘light-quantum’ hypothesis, Bohr’s quantum theory of the atom, Louis de Broglie’s wave-particle duality,* Erwin Schrödinger’s wave mechanics, and Werner Heisenberg’s uncertainty principle.

  I thought my head was going to explode.

  Mechanics is that part of physics concerned with the how and why of stuff that moves, governed by one or more mathematical equations of motion. In hindsight, our problems were compounded by the fact that the evolution of our understanding of classical mechanics had stopped with the school textbook version of Newton. We were not being trained to be physicists, and so missing from our education was the elaborate reformulations of classical mechanics, first by Joseph-Louis Lagrange in the eighteenth century, and then by William Rowan Hamilton in the nineteenth. These reformulations weren’t simply about recasting Newton’s laws in terms of different quantities (such as energy, instead of Newton’s mechanical force). Hamilton in particular greatly elaborated and expanded the classical structure and the result, called Hamiltonian mechanics, extended the number of situations to which the theory could be applied.

  We were therefore confronted not only with this extraordinary thing called the quantum wavefunction, but also with the challenge of writing down something called the ‘Hamiltonian’ for a specific physical system or situation, such as the orbit of an electron in an atom or the vibrations of a chemical bond holding two atoms together, without really understanding where either of these things had come from.*

  But, make no mistake, I was completely hooked. I filled my notebooks with equations that looked…. well, they looked beautiful. I still didn’t really understand what any of it meant, but I learned how to use quantum mechanics as best I could and set aside any concerns. I went on to complete a doctorate at Oxford University and a couple of years of postdoctoral research at Oxford and at Stanford University in California, before returning to England to take up a lectureship in chemistry at the University of Reading. Although I was never blessed with any great ability in mathematics, I learned a great deal more about quantum mechanics from Ian Mills, professor of chemical spectroscopy in my department, and I take some pride in a couple of research papers I published on the quantum theory of high-energy molecular vibrations.

  Then, in 1987, whilst working for a couple of months as a guest researcher at the University of Wisconsin-Madison, I happened upon an article that sent me into a tailspin. This was written by N. David Mermin.1 It told of something called the Einstein–Podolsky–Rosen ‘thought exp
eriment’, which dates back to 1935, and some laboratory experiments to probe the nature of quantum reality that had been conducted by Alain Aspect and his colleagues in 1982.

  I felt embarrassed. I had come to this really rather late. Why didn’t somebody tell me about all this before?

  I had allowed my (modest) ability in the use of quantum mechanics to fool me into thinking that I had actually understood it. Mermin’s article demonstrated that I really didn’t, and marked the beginning of a 30-year personal journey. I’m now the proud owner of several shelves overflowing with books on quantum mechanics, science history, and philosophy, and a laptop filled with downloaded articles. I’ve written a few books of my own, the first published in 1992.

  I can happily attest to the fact that, like charismatic physicist Richard Feynman, I still don’t understand quantum mechanics.2 But I think I now understand why.

  * More than forty years later, I can still hear my lecturer pointing out as an aside that de Broglie is pronounced ‘de Broy’.

  * I’ve written a technical book, suitable for readers with a background in physical science and some capability in mathematics, called The Quantum Cookbook: Mathematical Recipes for the Foundations of Quantum Mechanics. This was published by Oxford University Press in 2020 and I consider it a ‘companion’ to this volume. It is the book that I would have found really helpful when I was 18.

  Part I

  The Rules of the Game

  1

  The Complete Guide to Quantum Mechanics (Abridged)

  Everything You’ve Ever Wanted to Know, and a Few Things You Didn’t

  Here’s what I’ve learned over the past forty years or so.

  Nature is lumpy, not smooth and continuous

  We now know that all matter is composed of atoms. And each atom is in turn made up of light, negatively charged electrons ‘orbiting’ a nucleus consisting of heavy, positively charged protons (two up quarks and a down quark), and electrically neutral neutrons (one up quark and two down quarks).* Atoms are discrete. We can say that they are ‘localized’. Atoms are ‘here’ or ‘there’. In itself this is not particularly revelatory.