The Quest for Quantum Gravity: The Stubborn Offspring of Quantum Field Theory and General Relativity

Meet the Parents of Quantum Gravity: Quantum Field Theory and General Relativity

The Quest for Quantum Gravity: The Stubborn Offspring of Quantum Field Theory and General Relativity: imageQuantum Field Theory (QFT) and General Relativity (GR) form the theoretical and mathematical foundations for modern physics and cosmology.  QFT is an extension of Quantum Mechanics (QM), accounting for creation and annihilation of particles.  The primary entities in QFT are fields rather than particles, and it can be expressed in a Lorentz-invariant form, consistent with Einstein’s Special Theory of Relativity.  GR is, of course, Einstein’s brain child that explains gravity as the curvature of space and time, induced by matter and energy.  GR enabled Einstein to correctly calculate the magnitude of the precession of Mercury’s perihelion and the deflection of light by the Sun, and almost enabled him to predict the expansion of the cosmos.

These two paradigms, QFT and GR, have enjoyed unprecedented success in their range of validity, precision of experimental verification, and the amazing technologies that they have made possible.  However, many questions remain unanswered.  Puzzles include: what was the physics of the early universe and the pre-universe, what is dark matter, what is dark energy, what is the origin and nature of spacetime, what goes on at the horizon of a black hole and at a black hole singularity, how can gravity be united with the other three forces in a unified theory, what is the role of gravity in quantum decoherence?  Answering these questions may require finding a more general theory that merges QFT and GR into a unified framework encompassing both paradigms, a theory known as Quantum Gravity (QG).

QFT is essentially the theory of the very small, where quantum effects dominate and gravity can be ignored because it is so weak. GR is essentially the theory of the very large or heavy, where gravity dominates and quantum effects disappear.  A theory of QG must be able to predict and explain situations where both quantum effects and strong-field gravity are important.  Quantum Gravity in under five minutes:

The Apparently Incompatible Natures of Quantum Gravity’s Parents

QFT and GR are founded on seemingly different premises for how the universe works.  For example, in QFT, particle fields are embedded in the flat (Minkowski) spacetime of Special Relativity.  In GR, time flows at different rates depending on the spacetime geometry.  And gravity is due to the curvature of spacetime, which changes as gravitational masses move.  The most straight-forward ways of combining the two theories by quantizing gravity are non-renormalizable.  This means that calculations run away to infinity and cannot be tamed through a redefinition of certain parameters, as is done in QFT.

This problem is related to the fact that all particles attract each other gravitationally, and energy as well as mass create spacetime curvature.  When quantizing gravity, there are infinitely many independent parameters needed to define the theory.  At low energies, this form of quantum gravity reduces to the usual GR.  But, at high energies (small distance scales), all of the infinitely many unknown parameters are important and predictions become impossible.

A workable theory of quantum gravity must make use of some deep principle that reduces the infinitely many unknown parameters to a finite and measureable number.  Attempts at a workable theory of quantum gravity include string theory, loop quantum gravity, non-commutative geometry, causal dynamical triangulation, and a holographic universe.  Of course, which hypothesis you prefer is not a decision to be taken lightly:

The challenge of uniting QFT and GR is further compounded by the lack of experimental results that could point to a breakdown of either QFT or GR; or results from experiments that are sensitive to both theories.   Scientists are turning to a variety of astrophysical as well as table top experiments to address this issue.

Searching for Common Ground Between Quantum Field Theory and General Relativity

Testing the predictions of quantum theory on macroscopic scales is one of the outstanding challenges for modern physics.  Some experiments are not tests of a specific theory of quantum gravity, per se.  Rather, they look for a deviation from some fundamental tenet of either QFT or GR, with the hope that this will guide theorists in how to supplant either QFT or GR.  Other experiments attempt to create or observe conditions that are sensitive to both theories, to see how they play together.

Common to many philosophical or phenomenological approaches to QG is the possibility that fundamental symmetries, essential in our current understanding of the universe, may not hold at extremely small distance scales or high energy scales, due to a discrete structure of spacetime. Or, perhaps these symmetries do not hold in a highly curved spacetime with boundaries, such as in the vicinity of a microscopic black hole or the cosmological horizon of an inflationary universe.

These symmetries include Lorentz Invariance (LI) and CPT symmetry (charge conjugation – parity transformation – time reversal).  Lorentz invariance means that a property or process remains invariant under a Lorentz transformation. That is to say, it is independent of the coordinate system and independent of the location or motion of the observer, and the location or motion of the system.  CPT symmetry requires that all physical phenomenon are invariant under the combined operations of charge conjugation (swapping matter and antimatter), parity transformation (reflection in a mirror), and time reversal (viewing the process in reverse).

The IceCube South Pole Neutrino Observatory has weighed in on this issue, setting extremely tight limits on a possible violation of Lorentz Invariance.  Neutrinos, lacking strong or electromagnetic interactions and moving at essentially the speed of light (due to their teeny, tiny, and as-yet un-measureable, mass), are sensitive probes of these effects.  IceCube uses data from interactions of high energy atmospheric and astrophysical neutrinos in the South Pole ice.  See Search for a Lorentz-violating sidereal signal with atmospheric neutrinos in IceCube”, Stringent constraint on neutrino Lorentz-invariance violation from the two IceCube PeV neutrinos, and Probing Planck scale physics with IceCube.

The Fermi Gamma-ray Space Telescope is also a member of this club, using photons rather than neutrinos: Constraints on Lorentz Invariance Violation with Fermi-LAT Observations of Gamma-Ray Bursts and Constraints on Lorentz Invariance Violation from Fermi -Large Area Telescope Observations of Gamma-Ray Bursts.

Another sweet spot is the equivalence principle (EP), which provides the foundational basis for GR. The EP is the idea that the effects of acceleration are indistinguishable from the effects of a uniform gravitational field. The EP requires that gravitational and inertial mass are equivalent; that a particle’s coupling to a gravitational field is equal to its inertial mass.  See, for example, Expanded solar-system limits on violations of the equivalence principleor A millisecond pulsar in a stellar triple system.

Foundational Principles of Quantum Mechanics and the Cosmic Microwave Background

I have previously discussed the resurgence of de Broglie-Bohm mechanics, despite its historical neglect, in Hydrodynamic Quantum Analogs”.

In Beyond the Quantum, Antony Valentini follows the logical consequences of Louis de Broglie’s pilot wave theory to predict evidence of quantum non-equilibrium in the Cosmic Microwave Background (CMB).   Pilot-wave theory makes use of hidden variables.  The canonical interpretation of quantum mechanics says that there are no well-defined trajectories.  But in pilot-wave theory, these hidden variables describe the trajectories for whatever particles or fields a system may contain.  They can also explain the apparently random outcomes of quantum measurements.

Pilot-wave theory gives the same observable results as conventional quantum theory if the hidden variables have a particular distribution, a quantum equilibrium distribution, analogous to an ensemble of particles being in a thermal equilibrium.  But, as Valentini points out, there is nothing in de Broglie’s dynamics that requires this assumption to be made.  When the hidden variables have an equilibrium distribution, superluminal signaling is not possible; any attempted non-local signals would average out to zero.  However, if the hidden variables are not in an equilibrium distribution, superluminal signals may become controllable and observable! Relativity theory would be violated; time would be absolute rather than relative to each observer!

To help understand this, Valentini provides an analogy with classical physics:

“…For a box of gas, there is no reason to think that the molecules must be distributed uniformly within the box with a thermal spread in their speeds. That would amount to restricting classical physics to thermal equilibrium, when in fact classical physics is a much wider theory. Similarly, in pilot-wave theory, the `quantum equilibrium’ distribution – with particle positions distributed according to the Born rule – is only a special case. In principle, the theory allows other `quantum non-equilibrium’ distributions, for which the statistical predictions of quantum theory are violated – just as, for a classical box of gas out of thermal equilibrium, predictions for pressure fluctuations will differ from the thermal case. Quantum equilibrium has the same status in pilot-wave dynamics as thermal equilibrium has in classical dynamics. Equilibrium is a mere contingency, not a law.

…It seems natural to assume that the universe began in a non-equilibrium state, with relaxation to quantum equilibrium taking place during the violence of the Big Bang.

…The crucial question is whether the early non-equilibrium state could have left traces or remnants that are observable today.”

Quantum non-equilibrium at the onset of inflation would modify the spectrum of anisotropies (differences from place-to-place) in the CMB sky.  Hence, measurements of the CMB can test for the presence of quantum non-equilibrium during the inflationary phase.

See also: Samuel Colin and Antony Valentini, Mechanism for the suppression of quantum noise at large scales on expanding space, where the authors present numerical simulations showing how the expansion of space can slow down the relaxation to quantum equilibrium in the super-Hubble regime:

“Given these results it is natural to expect a suppression of quantum noise at super-Hubble wavelengths. Such suppression could have taken place in a pre-inflationary era, resulting in a large-scale power deficit in the cosmic microwave background”.

A variety of tests of fundamental physics, conceivable with artificial satellites in Earth orbit and elsewhere in the solar system, are discussed in David Rideout, et al., Fundamental quantum optics experiments conceivable with satellites — reaching relativistic distances and velocities:

“We propose to push direct tests of quantum theory to larger and larger length scales, approaching that of the radius of curvature of spacetime, where we begin to probe the interaction between gravity and quantum phenomena. …the potential to determine the applicability of quantum theory at larger length scales, eliminate various alternative physical theories, and place bounds on phenomenological models motivated by ideas about spacetime microstructure from quantum gravity.”

Table-Top Tests of Quantum Mechanics and General Relativity

The question of simultaneously observing the effects of quantum physics and GR in a table-top experiment can be framed as simply as this: The idea that particles can be in superpositions of multiple states (states with different trajectories, different spins, different energies, etc.) is an essential feature of quantum mechanics.  If a particle is in a superposition of states with different paths through a gravitational field, for example, the different superpositions should be effected differently by the different trajectories through spacetime.  If a particle is in a superposition of different energy states, these different superpositions should create different gravitational fields.  If a macroscopic object could be placed in a superposition of oscillating and non-oscillating, for example, its gravitational field should also split into a superposition.  What does a superposition of gravitational fields look like and how does it behave?

Unfortunately, quantum superpositions are very delicate. As soon as a particle in a superposition interacts with the environment, it appears to collapse into a definite state (see Decoherence and the Quantum to Classical Transition; or Why We Don’t See Cats that are Both Dead and Alive).  Only tiny particle-sized entities can be maintained in quantum superpositions for any significant period of time.  However, only macroscopic objects have detectable gravitational fields.  So this presents immense technical challenges for experimentalists.  People are working very hard to improve upon these limitations.  See, for example, Brian Pepper, et al, Optomechanical superpositions via nested interferometry and Macroscopic superpositions via nested interferometry: finite temperature and decoherence considerations.

Magdalena Zych and his colleagues are searching for evidence of gravitationally-induced time dilation and its effects on the phase of a quantum state: Quantum interferometric visibility as a witness of general relativistic proper time, (also available here).  They propose using a Mach-Zehnder interferometer (MZI) in a gravitational field.  According to GR, proper time flows at different rates in different regions of spacetime.  Their proposed experiment requires a particle with evolving internal degrees of freedom, such as spin or internal vibrations, that can act as a clock.  And the two different legs of the MZI are at different gravitational potentials.

Table-Top Tests of Quantum Mechanics and General Relativity: image

Difference between probabilities to find the particle in different outputs of the Mach–Zehnder interferometer as a function of the time ΔT for which the particle travels in a superposition of two trajectories (corresponds to changing the length of the interferometric arms). Without the ‘clock’ degrees of freedom, the dashed, black line would be the result. With the ‘clock’ and the predictions of GR, the predicted result is the blue line. From “Quantum interferometric visibility as a witness of general relativistic proper time”.

If there is a difference in proper time elapsed along the two legs of the interferometer, the particle’s internal clock will evolve into two different quantum states.  This is a consequence of the prediction that the clock ticks at different rates when placed in different gravitational potentials.  As a result of the quantum complementarity between interference and which-path information (in the form of the different internal clock values), the general relativistic time dilation will cause a decrease in the interferometric visibility (see the adjacent figure).

“Such a reduction in the visibility is a direct consequence of the general relativistic time dilation, which follows from the Einstein equivalence principle. Seeing the Einstein equivalence principle as a corner stone of general relativity, observation of the predicted loss of the interference contrast would be the first confirmation of a genuine general relativistic effect in quantum mechanics.”


This has been just a sampling of the work underway to pry nature’s secrets from her grasp.  For theorists and experimentalists, working on the interplay between QFT and GR with the ultimate goal of creating a theory of QG, is one of the most challenging and stimulating areas of research in fundamental physics.  If this brief discussion has piqued your interest, let me know.  I can point you towards more resources concerning the theoretical and experimental work taking place on the road to quantum gravity.

Three Roads to What Lies Beyond Quantum Mechanics

Alternative Paths to an Interpretation of Quantum Mechanics

Einstein and Bohr discussing Quantum Mechanics image

This is the real reason why we have not been able to discover a theory more fundamental than quantum mechanics. We no longer wear three piece suits to physics conferences.

If you have read my earlier posts, you have likely concluded that I am a realist with respect to the foundations of quantum mechanics (QM).  I believe there is some deeper reality behind the equations of QM.  This deeper reality, or more fundamental theory, will account for the apparent nonlocality that we see in quantum events.  It will also explain the intrinsic randomness in quantum theory, and how nature decides which option to choose.  In my opinion, the wave function (or state vector) represents more than just our knowledge of a system.  However, it may be just an approximation for the correct representation or description of reality.  I also believe that our preconceived notions of reality will need to be altered, just as they were when Einstein developed the Special (SR) and General (GR) Theories of Relativity.

My views contrast with those who believe that QM is the ultimate mathematical formulation.  Among those that believe QM is complete, there are two classes – those that believe it is not even appropriate to ask “why”, “how”, or “what”; and those that seek a set of principles that lead directly to QM.  Members of the first group hold that there is no such thing as objective reality.  Their position is that QM merely encodes what we know (or can not know) about the quantum world.  All we can talk about are the results of measurements that we make.  They argue that it is not even appropriate to ask nature to reveal a deeper understanding, because there is not one.  The second group looks for a deeper understanding, but they look for this conceptual or philosophical foundation within the framework of the current quantum theory.  I think both of these approaches are flawed.  In my opinion, the first approach amounts to forfeiture and the second is a dead end.  To move beyond QM and find a deeper conceptual understanding of the universe, we need to re-zero our preconceived notions and come at the problem from a new direction.

A search of the internet, or YouTube in particular, reveals multitudes of blogs and videos about QM.  There are many high quality documentaries and video clips, with excellent production qualities.  But, the vast majority parrot the ideas that “spooky action at a distance”, entanglement, wave function collapse, the mysterious role of an intelligent observer, etc., are intrinsic magical mysteries of the universe.  They give the impression that our knowledge of the quantum world is secure, and that we might as well accept this bizarre status quo.  Worse, I have seen expert speakers on these documentaries claim that it is not even correct to question these aspects of QM, these are not the “correct questions to ask”.  Metaphysical BS such as “each of us constructs our own version of reality” really gets me going.  No, we are not constructing alternate realities in our heads.  Each of us is constructing a different, flawed and approximated, representation or model of reality in our heads (some more flawed and approximated than others).  But this is not the same as creating a separate reality.  I believe there is some higher level theory, of which QM is an approximation, that will explain these issues.

Quantum Theory as a Labor of Love (Vice Money)

There are several theoretical and experimental physicists doing work on interpretations of QM.  It is hard work.  Generations of physicists have gone before you and have not been able to understand quantum theory.  Additionally, you are working up hill against a field that is only slowly realizing the importance and applicability of your work.  Funding is hard to come by, as you compete with paradigms that have sapped man power and resources for decades.  Frustratingly, you are forced to compete in isolation with these fields despite their lack of actual physics results.  We need more people, and more funding, looking into the foundations and extensions of quantum mechanics.  Back in the 1980’s, many physicists believed that an end run had been found.  They believed that string theory was the theory of everything, from particle physics to cosmology, from the quantum world to black holes.  All the hype has not manifested itself in reality.  It is time to re-tool theoretical physics.  This may mean taking smaller steps, ensuring that we understand where our footing is at each hurdle.

The potential payoffs are huge.  This includes understanding and resolving open issues in Quantum Field Theory, which serves as the basis for the Standard Model of Particle Physics.  It may lead to a theory of quantum gravity, and explanations of dark matter and dark energy.  This would lead to a replacement for the Standard Model of Cosmology (the “Lamda Cold Dark Matter” model – beyond the scope of this post, perhaps more on that later).  These models that aspire to explain all of particle physics and all of cosmology have many free parameters.  The ability to tune these parameters to match the observations weakens our confidence in the uniqueness and correctness of these theories.  Hopefully, a new extension of QM will help fix some of these free parameters.

Getting More Specific About What Lies Beyond Quantum Mechanics

I am not a fan of models that simply add additional mathematical structures to the Schrödinger equation in order to reproduce observed behavior.  Modified Schrodinger dynamics is one example of such attempts.  I am also not a fan of interpretations that ask us to accept too much without conceptual reimbursement, such as the Many Worlds Interpretation (MWI).  In the MWI, each quantum event creates a new universe.  With countless quantum events occurring throughout the universe every moment, the instantaneous proliferation of fully developed universes is staggering, to say the least.

Sean Carroll is a talented and prolific physicist and a wonderful writer. See, for example, his book From Eternity to Here: The Quest for the Ultimate Theory of Time, also linked from my Recommended Reading page.  Dr. Carroll recently gave an interview on interpretations of QM:

I agree with his assessment that the failure of physics to develop a conceptual understanding of quantum mechanics is an embarrassment.  However, I differ with him over his endorsement of the MWI.  There are other interpretations that can resolve the paradoxes of the canonical (Copenhagen) interpretation of QM without such wild assertions and assumptions.  Examples of these alternatives include de Broglie-Bohm mechanics, the Transactional Interpretation, or the Two-State Vector Formalism.  This does not mean that one of these other interpretations are necessarily correct.  It does mean that we should not be so eager and willing to swallow MWI.  The MWI will have to provide a better return on my investment before I start to take it seriously.Many Worlds Interpretation of Quantum Mechanics_image

I have read several papers that attempt to derive quantum mechanics from invariance laws associated with information, probability, causality, contextuality, composability, etc., (name your principle).  While interesting, I just don’t have the sense that these approaches are on track to capture the essence of reality.  Perhaps the problem is that they are starting with the end in mind.  Although “begin with the end in mind” is one of Stephen Covey’s key tenets (The 7 Habits of Highly Effective People), in this case it may be a handicap.

Compare this situation with how Einstein discovered SR and GR.  Einstein was not trying to reproduce Newton’s absolute space and time or Newtonian gravitational theory.  He saw evidence in the world around him and in his thought experiments that led him to postulate a new invariance law.  For SR, it was the constancy of the speed of light.  For GR, it was the equivalence of gravitational and inertial mass.  He then followed these postulates to where ever they took him.  With his new theories, Einstein was able to explain experimental results that contradicted existing theories; the experiment by Michelson and Morley that failed to detect the ether and the anomalous precession of the orbit of Mercury, for examples.  More importantly, he was able to make additional testable predictions.

Stating the Goal: Discovering the Path to What Lies Beyond Quantum Mechanics

So, another approach is to discover some fundamental physical principle(s) that leads to some new theory.  String theory is an example of this.  State a hypothesis: things are made up of strings rather than points.  Then, see where that leads.  However, you have to be willing and able to recognize when your hypothesis is not productive.  Hopefully, the new theory that explains QM will make some testable predictions that contradict other interpretations of conventional QM, so that we can tell which is right and which is wrong.  As if that would not be awesome enough, an even more amazing discovery would be a theory that reduces to QM in one limit, and GR in another.

I want to stress that I think this more fundamental theory will likely be founded on principles that seem, at first blush, to not have much to do with classical QM or GR.  And the math may look very different until approximations are made.  There may be additional hidden variables that are not accounted for in the current set of equations.  Something totally new.  Extra dimensions perhaps.  Not necessarily extra space or time dimensions.  But, some extra freedom for the “force carriers”, for entanglement, or for whatever advanced and retarded wave functions are representing.

Maybe some applicable insights can be found in the Aharanov-Bohm effect, Significance of Electromagnetic Potentials in the Quantum Theory.  Aharanov and Bohm showed that, contrary to classical mechanics and classical electromagnetism, the electromagnetic four-potential can have an observable effect on charged particles, even in regions where the electric and magnetic fields cancel (where there are no forces on the particle).

Fields such as a gravitational field, an electric field, or a magnetic field can be described in terms of a potential.  In the case of electromagnetism, you need a scalar and a vector potential.  Gravitational fields require a tensor potential.  The fields can be derived from the potentials.  However, the potentials can not be uniquely determined from the fields.  That is getting more in depth than we should for the present.  Suffice it to say, however, that prior to the prediction (and subsequent experimental verification) of the Aharanov-Bohm effect, physicists believed that all physically relevant dynamics could be expressed in terms of the fields.

In the Aharanov-Bohm effect, an electrically charged particle is affected by an electromagnetic field despite being confined to a region in which both the magnetic field and electric field are zero. This is an apparent nonlocality of the field interactions.  However, it can also be interpreted as the coupling of the electromagnetic four-potential with the phase of a charged particle’s wave function.  The effect is observed in interference experiments, where the phase difference of two interfering wave functions (state vectors) modifies the interference pattern.  The Aharonov–Bohm effect shows that the electric and magnetic fields do not contain full information about the physics.  The electromagnetic four-potential offers a more complete description of electromagnetism than the electric and magnetic fields alone can.

Seeing Only the Shadows of the Quantum Universe

Like the prisoners in Plato’s Allegory of the Cave, what we see taking place at the quantum level may be just shadows of the true reality.

The title of this post, “Three Roads to What Lies Beyond Quantum Mechanics”, is a play on the title of Lee Smolin’s quantum gravity book: Three Roads To Quantum Gravity (Science Masters).