Quantum mechanics: weirder than we can think
Local realism is dead, but we don't really understand what is left
Where this is heading
Quantum mechanics attracts more mysticism than any other aspect of physics. I want to explain what’s known and what isn’t for non-technical readers. I hope that the explanation will clear away some shallow underestimations of how strange QM is while protecting against some of the equally shallow mystical woo that circulates.
The tradition is to present the development of quantum mechanics quasi-historically. The disadvantage of that approach is that one tries to picture the new phenomena in old frameworks, and these pictures tend to persist. So I'm going to follow a different path. First we'll just smash the old framework. Then we'll look at the weird structure that arises from the ruins. Finally, we’ll have a look at the core unsolved problem.
Warning: You may feel some discomfort.
A story to show what you believe is wrong
What do we all believe about the world? Einstein thought it was deterministic, that "God doesn't throw dice", but we won't start with any strong opinion like that. We'll start with something much milder, from a paper by Einstein, Podolsky and Rosen:
If, without in any way disturbing a system, we can predict with certainty (i.e., with probability equal to unity) the value of a physical quantity, then there exists an element of reality corresponding to that quantity.
So we aren't insisting that nothing be pure chance, just that things which are fully predictable have some actual physical cause. Furthermore, the cause must be located where the effect occurs, not somewhere else or at some point in the future. Borrowing from our discussion of relativity, that means that the causes can't travel around faster than the speed of light. Otherwise, according to perfectly good points of view, they'd travel backwards in time and lead to all sorts of sci-fi paradoxes.
The assumptions I've just described are called "local realism." In our gut, we all believe them. Now I'll describe how we know they are false.
People's eyes glaze over if we talk about unfamiliar particles. So I'll tell a silly story about people. Its logical structure is absolutely identical to the true story we can tell about some smaller stuff.
Say you notice on a moon (your planet has 6 moons) some guy who every second raises one hand or the other. So far as you can tell, he does it randomly. Is it really random, or does he have some hidden system? Looking directly, there's no way to tell. Fortunately, you have two eyes, and can look through trick binoculars at the moon directly opposite his. (No movies are allowed.) There's another guy there, and he raises the same hand each time. There's no time for signals to get from one guy to the other, so they must have got their patterns coordinated ahead of time. You can predict one based on the other, so they are each determined by some "element of reality." Maybe they both came from earth where they picked up matching instructions. There are 3 pairs of moons, and each pair acts this way whenever you check it. So all the guys have "elements of reality" (memorized lists?) telling them which hand to raise when.
Now let's just look at one moon from each pair. We'll call them A, B, and C. With only two eyes, you can only see two at once. When you look at A and B, you find that 85% of the time, they raise the same hand. So their lists must differ 15% of the time. When you look at B and C, you find the same 15% difference.
What do you expect if you now look at A and C? Maybe those 15%'s that each differed from B were the same, so A and C will always agree. Maybe those 15%'s didn't overlap, so A and C will disagree 30% of the time. Or it could be anywhere in between. Simple.
So now you look at A and C. They differ 50% of the time. Impossible. The A and C lists can't both be almost the same as the B list and yet so different from each other.
So those lists couldn't really exist. Maybe only parts of the lists exist, the parts that have instructions for the moons you're looking at. Maybe somehow these moon-men knew which moon pair you'd look at when. So you try switching around which pair of moons you look at each time. Sometimes you use what feels like "free will" to pick a pair of moons. Sometimes you roll dice. Sometimes you use random number generators in your calculator. None of it makes any difference.
Could there be a complete conspiracy, where your "free will", your dice, your random number generators, etc. were all controlled by the same entity that decided which random-looking lists to hand out to the moon men? Sure, we can't rule that out, but we don't believe it.
So if our story were to be true, we'd have shown that the world cannot be described by any non-conspiratorial local-realist picture. Somehow, the world "knows" that those guys on opposite moons must raise the same hand even though it has absolutely nothing in it to set which hand it will be. Of course, this story is nonsense.
Nevertheless, many, many stories with identical logical structure, but involving some small-scale events, are simple experimental results. One of my colleagues has even set up a version of this experiment for a routine undergrad lab. Undergrads can show violations of the "Bell Inequalities" (one of which you saw illustrated above), the relations derived in the 1960's by John Bell to describe the limits on the behavior of non-conspiratorial local realism.
Non-conspiratorial local realism is false. So when we start to describe the positive content of quantum mechanics and you find yourself saying "Oh, maybe it's like....", stop yourself. Are you likening it to something that obeys local realism? If so, that ain't it
Quantum Mechanics
Let’s look at a little of the contents of quantum mechanics, the framework now used to describe everything in the physical world except gravity. Although there's no doubt about how to use quantum mechanics as a nuts-and-bolts working theory, the interpretation remains open. The story will be a bit complex, and, unlike the previous discussion of why local realism is false, it will not be possible to stick strictly to points on which there's virtually unanimous agreement. Any description uses some vocabulary with arguable implications for the interpretation, but I'll try to label any disputed points. I'll leave many open questions, in the hope that a few of you will struggle through far enough to raise objections and questions yourselves.
In classical mechanics (including the relativistic kind) there are precise mathematical rules describing how the state of the world changes as a function of what its current state is. Many of you have seen an approximate piece of that, F=ma, in which the velocity of a particle of mass m changes (via acceleration a) as a function of the known features of its environment, the nearby things responsible for the force F. If you put together all the pieces, the collection of these rules determines the future based on the current state of things. There are two types of ingredients in that state. One is a collection of particles, things that have positions and velocities as well as other properties. The other is a collection of fields, like the electrical field, things which permeate all space, although with varying strengths. The particles give rise to fields, and the fields exert forces on the particles, in a big self-consistent system. We just saw that no such picture can describe our actual universe.
In quantum mechanics, things are in some ways simpler. Despite what you may have heard, there is only one type of ingredient, space-permeating fields. Things like electrons, which you may have seen pictured as little dots, never behave like anything except spread-out fields, although the degree of spreading can vary. One can write the current state of things as a field (sometimes wave-like) in space representing not just one type of particle but all sorts of different interacting types. One odd feature, however, is that this field doesn't have definite values for variables that you think ought to have definite values. Objects are always smeared out in space. They also always have a smeared out range of velocities. Except in the most boring cases (cases where nothing is happening) they have a range of energies. They typically have a range of particle numbers.
Despite that unfamiliar feature, in quantum mechanics we still have a rule giving exactly how the state is changing in time as a function of what the state currently is. The rule is:
ihd|Ψ>/dt=2πH|Ψ> .
Here |Ψ> is just the name of the state, d|Ψ>/dt is the rate at which it is changing in time, and H stands for a linear function of the state. "i" is the square root of -1, and the rest of the terms are boring constants. Don't worry if that looks confusing, because we're only going to use one simple feature of it.
So far, things look clear enough. You've heard about the random side of quantum mechanics, but we have a plain old equation with no randomness. You've heard about the dreaded Uncertainty Principle, but so far we just have a little spread in some fields. There's nothing more uncertain about that than the way a ripple in a pond certainly spreads out. Our equation is even local, meaning that the changes in |Ψ> can be calculated just from the part of |Ψ> nearby in space, so there's nothing spooky about it. So what's the big problem, other than that things are a little more spread out than you might have expected?
We know that trouble is coming, because that was a local theory, and we saw experimentally that no local theory can describe what we see. The first problem comes from the simplest feature of our equation, its strict linearity. The important part about linear equations (and this is really just simple school math) is that they obey superposition, meaning that if you have two different solutions you can add up some of one and some of the other and still get a solution to the same equation.
Why is that nice feature a problem here? Let's say you have a quantum thingy, say a single blip of light, which either makes it through a polarizer or not, depending on which way it's polarized. If it's horizontal (H), it gets blocked and nothing happens. If it's vertical (V) it gets through, triggers a detector, and starts a mechanism that kills a cat. (sorry for the traditional nasty example.) What's fishy about that? It's easy to prepare that blip polarized somewhere between H and V. The quantum state representing that is a combination of some H and some V. The H part was the start of a solution in which the cat lived. The V part was the start of a solution in which the cat died. So our exact superposition says what this solution looks like: the output is part dead cat and part live cat.
Wait! That's crazy. That's not what you see. So let's go beyond the cat to ask what the equation says you should see. Just write out the quantum state for the blip, the apparatus, the cat, and you. The output is a combination of a you seeing the live cat and a you seeing the dead cat. It doesn't equal your perceived experience.
To avoid misunderstanding, you should realize that such "quantum measurement" situations are not just contrived lab artifacts. We're bathed in them constantly- UV light blips interacting with your DNA, leading to a superposition of a you with a dead skin cell and a you dying from melanoma, etc. That also doesn't correspond to your perception.
What happens in our experience? We don't experience that whole output state. We experience one piece of it, a piece where big things (cats, people) have nearly definite properties. So this is where probability and uncertainty come in. We don't know which piece of the output state will describe our experience. It turns out we can correctly calculate the probabilities of each of the possible outcomes from a measure of how much of the quantum state headed toward that outcome.
If you have a spread-out electron wave hitting a tv screen, the quantum state is initially certainly spread out. When you see a blip of light from that wave hitting the screen, it comes from some particular region. The wave just gave you probabilities for what you would see. What had been a simple spread is converted to an uncertainty by whatever process causes us not to experience the whole output state predicted by our equation.
At this point, anybody in their right mind (if any are still reading) would be thinking that there must be something else, some little hidden clue to tell the blip where to show up, to make the cat live or die. This is where our first argument comes in. Any such "hidden variable" hypothesis has implications, and those implications are experimentally false.
Then you might think, ok somehow nature throws some secret dice to tell that electron where to show up, but here's where it gets stranger. When little objects interact, their quantum states become "entangled". Then when one of the possibilities for one of the objects becomes the actual outcome you see, that tells you what actual outcome you'll see for the other object, no matter how far away it is. As we saw before, there's no way to put little quantum dice at each spot and reproduce the observed behavior, because there's no way for the remote places to coordinate their results, and the actual results are coordinated.
Now may be a good point to remind you that this whole crazy story provides the tools for doing the calculations at the heart of modern physics and chemistry. The calculations are often extraordinarily precise. We know how to calculate the possibilities and what probabilities to assign them. For all practical purposes, we don't really need to know more about how this works.
Nevertheless, we would like some more understanding, some feel for what's up even if it's weird. At least a little start is provided by an analysis of something called "decoherence", beyond the scope of this blog. Decoherence explains how interactions with the outside world cause a quantum state of some region to break up into parts which quit showing wave-like effects with each other. Decoherence is already implicit in our basic story, and in many cases it can be explicitly calculated. (That's done a lot, since it's decoherence that makes it hard to build quantum computers.) One decoherent part would have a live cat and a you seeing a live cat, another decoherent part would have a dead cat and a you seeing a dead cat.
That still leaves some very big open questions, including:
1. Is there some sort of collapse of the quantum state to just one outcome? We usually speak as if this were how things happened.
2. Do all the parts still exist, just losing contact with each other? That's called the Many Worlds interpretation.
3. Was there some sort of "pre-collapse" in which there really was only one actual value of the coordinates? That's called the Bohm interpretation.
4. Is the traditional Copenhagen Interpretation correct?
There are other questions and other interpretations beyond the ones I discuss. This should give you some of the flavor. You won't find the answers here.
(1) implicitly invokes some sort of process outside the known equation. Attempts to make an explicit version have not yet succeeded. One central problem is the one we've mentioned- how do remote entangled objects get their random stories straight? That especially runs into problems with relativity, which gives paradoxes if information travels faster than the speed of light.
(2) is in some ways the most obvious interpretation. It has the advantage of not having to explain how nature's choices at remote locations get coordinated, because nature isn't making any choices. Everything happens. The beauty of not adding any extra processes has a drawback, however. There's no extra step where you can sneak in the rule for what the probabilities are. In my opinion no one has given a solid explanation of how it leads to the correct values for the observed probabilities, called the Born rule.
(3) used to look promising before the non-locality results were in. Now instead of having a little coordinate dot for each particle, we need one dot in some high dimensional space representing every particle in the universe. That doesn't seem to clear anything up.
(4) is still what many textbooks advocate, but I'm not sure what it means. I don’t think the textbook writers are either.
My speculations on the probability rule
The Many Worlds interpretation seems to be the cleanest, in that it adds no extra mathematical glop to the wonderfully successful underlying quantum theory. In recent years its popularity has steadily grown. Although outsiders often object that it offends Occam’s razor by postulating extra unobserved branches of the quantum state, it does that to avoid postulating extra unobserved mathematical modifications to a beautiful and extraordinarily accurate theory. Worlds are cheaper than equations. The argument has been nicely described in Sean Carroll’s recent book Something Deeply Hidden.
There is, however, that big catch that I mentioned– where does the probability rule that we observe come from? There have been many formal attempts to show that the observed Born rule probability, proportional to quantum measure, is the only possible one. All of these arguments share an implicit or explicit assumption– that the probability rule must lead to comprehensible, sensible results, not a truly unpredictably unpredictable mess. Carroll fortunately has expressed the underlying assumption in brief clear non-technical terms, describing a particular situation in which there are three possible outcomes, each with equal measure:
If we don’t want the probability of one branch to suddenly change when something happens on another branch, that means we should have assigned probability 1/3….exactly as the Born rule would predict.
It’s true that for us to have any sort of predictable world “we don’t want the probability of one branch to suddenly change when something happens on another branch”, but nature is not obliged to avoid what “we don’t want”. One cannot derive the implications of a theory by assuming that they must be what one wants.
Here’s a historical analogy that captures the point I’m trying to make. Classical thermodynamics led to a rule for how much thermal energy would end up in each simple vibration mode in equilibrium– classical equipartition. Applied to what should be the simplest modes, electromagnetic radiation in space, equipartition had a disturbing result, that the radiation energy density would be infinite. That’s a result “we don’t want”. Simply by assuming what we want, that the energy is finite, one can easily show that the radiant energy density must be proportional to the absolute temperature to the fourth power. It is! The problem was that the nice finite-energy assumption made no sense in terms of the underlying theory.
It was largely through struggling with that logical issue, the “black-body radiation” problem, that deep changes to the underlying picture were discovered. The changed picture is called quantum mechanics. Assuming that the results of one’s current theory must be nice and well-behaved regardless of what the theory actually says was a recipe many followed for not discovering quantum mechanics.
Is there a way without modifying the underlying math to obtain the observed Born probabilities, where the probabilities don’t change due to things happening to unobservable other branches? Maybe. Jacques Mallah has proposed a way that Born probabilities could emerge for a particular category of starting quantum states. So far as I know, he and I are the only ones who take it seriously. Anybody who wishes to take a deep dive into those speculations might take a look at a brief Berkeley talk I gave on the topic. As always, criticisms are welcome.