Science often proceeds one grave at a time

I just finished this tremendously readable and provocative #book: Escape from Shadow Physics.

The book is about foundations of quantum mechanics. To be precise, it’s about why we should not give up on trying to find a deeper theory of quantum mechanics.

The usual story in QM is that the theory is complete and the probabilistic nature of our measurements is just the way nature is. Einstein famously rebelled against this, and didn’t accept it even until his death.

Believe it or not, this attitude of his — “God does not play dice” — is in minority today. Most of the modern physicists believe that QM is complete, and that at the micro level reality is simply probabilistic when you make observations. In other words, there’s no deeper explanation to why we get statistical outcomes in QM experiments.

Of course, this is so deeply unintuitive that there’s a cottage industry of interpretations for why is this the case. From multiverse to consciousness collapsing the wave function to multiple past timelines. All this would be super strange and crazy had QM not been confirmed experimentally to a high degree of accuracy.

So, what is to be done? Should we just “shut up and calculate” and never ask what’s going on beneath the quantum levels.

---

Is quantum mechanics final?

The book is actually both a history of quantum mechanics but also a disguised guidebook on the history and philosophy of science. It goes back in time and talks about multiple accounts for when respected scientists suggested something was impossible, but later they turned out to be wrong.

A striking example is that of heat. Fourier modeled its flow with an equation and that led many prominent scientists to believe that heat was a fluid. This made the alternative explanation — the kinetic theory of heat comprising atoms smashing with each another — open to getting attacked. Of course now we know what heat is at a much deeper level than anyone at that time could have anticipated; we now know heat is due to kinetic energy of atoms. Is the case similar with QM?

Proponents of QM are quick to point to Bell’s Theorem — a supposedly “final” proof that there’s no deeper explanation to QM and that the reality it describes is just the way nature really is?

The author does a great job of convincing how many such “impossibility” theorems often have hidden assumptions that are not fully elucidated that well render them weaker. What an impossibility theorem often shows is that the assumptions implicit in it cannot be violated. But there is often a lot of detail lurking in reality which we can’t fully anticipate in our theorems.

So, it is best to view impossibility theorems as mathematical theorems and not necessarily a statements about physical reality.

---

I think the book is hugely inspiring.

I agree with its central message: we should allow dissenting voices that argue against finalities in our understanding. We should celebrate the fact that there are scientists who are not satisfied with the status quo.

May future generations discover there are indeed “hidden variables” lurking in reality which fully explain the apparent statistical outcomes we observe in our quantum experiments!