Correctly Predicting Failure

It’s time for scientists to get loud

Written by Dr. Jeff Hester. Modified on May 11, 2020. Posted in Thoughts

Now is not the time for scientists to be circumspect and silent. We are on the short end of a battle over whether truth even matters. If scientists do not stand up for what is real, who will?

I told you so.

In mid April I posted an article pointing out that the assumptions at the heart of the oft-cited IHME COVID-19 pandemic models were unquestionably wrong. I went so far as to predict that within a week or so it would become obvious that the model was failing, and that the IHME forecasts were too low by about a factor of two. I got some pushback over that article. The most strongly worded was an email suggesting that I keep my thoughts to myself about anything other than “limited fields of some subjects in astrophysics.”

This is a really delightful look at some toy epidemiological models that capture a lot of the features of more sophisticated models, as well as show how and why things like social distancing, human mobility and improved hygiene have such an effect on the course an epidemic takes. Interestingly, adding sophistication to models tends not to change basic conclusions in fundamental ways. Adding sophistication does, however, capture more of the complexity of the system, leading to much greater uncertainty, instability and emergence of chaos.

The writer said he enjoyed my columns in Astronomy Magazine, so was clearly a person of refined taste. And as it happens the criticism was soon moot. Two weeks after I posted my article the IHME released a major update of their model. They acknowledged some of the same problems I had discussed, as well as others, and threw away the curve fitting approach that I critiqued. In the process they increased their predictions of summer COVID-19 deaths from 60,000 to almost 135,000.

The IHME deserves a nod of some sort for its efforts to bury the headline in academic-speak. Quoting from their update:

“This allows us to account for potential increases in transmission intensity if – or as the data increasingly suggest, when – social distancing mandates are eased and/or human mobility patterns rise. The latter is particularly important, as it appears that many populations are exhibiting increases in movement and thus possible interactions with each other, even in places where distancing policies remain in place.”

Translating from journalese to plain English, people are doing really stupid shit, and are going to die by the bucketful as a result. Further, people doing stupid shit in one place kills people even in other places where they aren’t doing so much stupid shit. And “if – or as the data increasingly suggest, when” people get busy doing even more stupid shit than they are now, we are going to look back on the possibility of only 135,000 deaths and sigh at the memory of our naïveté.

Science is a way of thinking, not a collection of specialized knowledge.

Even so, my critic’s basic question is reasonable. I’ve written about the nature of knowledge and cognitive errors like the Dunning-Kruger Effect and our craving for certainty. I am also aware that I am not beyond those temptations; I look for them regularly in the face in the mirror.

What the reader failed to recognize, I think, is that science is not a body of knowledge. Quite the contrary, science is a commitment to challenging knowledge. As a scientist when I look at something like a pandemic model (or just about anything else, to be honest), my first question is, “does that pass a smell test?” If it doesn’t, and if I care about the answer, I dig. In the case of the IHME models, I didn’t have to dig far. When I read their technical paper I was deep into a “WTF?” moment by page three. (I wonder how many people who read this column had similar experiences of their own.)

Back to my critic’s question, how could an astrophysicist working from his home office confidently and correctly predict the failure of what had been touted as the premier model of the COVID-19 pandemic? Was it a lucky guess? Or did it have something to do with a way of thinking along with a broad base of knowledge built and honed over four decades as a working scientist? Why might someone like me have better, more reliable insights into what is going on right now than, for example, your run-of-the-mill AK-47-toting Trump supporter intent on “liberating” Michigan?

Reality is reality is reality.

Those of you out there who are scientists know the answer. It’s handy to draw boundaries between disciplines when it’s time to Balkanize a university and start fighting over who gets faculty lines and new buildings. But the brain doesn’t come equipped with nice well-defined barriers that separate thoughts about physics from thoughts about epidemiology from thoughts about economics. Organization charts of universities can be Balkanized, but knowledge cannot.

Numerical Methods for Solving Partial Differential Equations

Regardless of whether you are tracking a pandemic or watching the universe evolve, the rules that govern the world describe how things change in response to other things. These expressions of change, typically expressed as partial differential equations, have to be summed up, or “integrated” to predict how a system will behave over time. Those systems of equations and the techniques used to solve them share much in common across all fields of science and engineering. Knowledge, experience and intuition gained in a field like astrophysics often transfers well to other fields like epidemiology.

For quite a while the really cool stuff has often involved people who are willing to run roughshod over those artificial divisions. The catchphrases for people who refuse to respect such boundaries include things like “interdisciplinary,” “transdisciplinary,” or just plain “innovative.” Quite often it takes the perspective of an outsider to see what the insiders are missing. Physics turns out to be a good jumping off place for such excursions, as witnessed by the fact that physicists have helped revolutionize everything from financial markets (sigh), to biology, to models of brain function.

In the real world math is math is math. Systems are systems are systems. Everything is ultimately physics. Lessons of complexity and chaos generalize. Lift the hood on epidemiological models and you find the same kinds of engines that power models of everything from astrophysical shock waves, to the earth’s climate, to the evolution of the structure of the universe. Just for the heck of it, I put “numerical modeling” into Google to see what came up. On the first few pages were links to application of the same basic techniques to geology, drug propagation in tissue, battle planning, brain function, development of coastal areas, oceanography, underground tunneling, drainage basins and estuaries, stability of offshore wind turbines, welding, forensic structural analysis, power plants, masonry of historical structures, the human eye, arterial networks, high temperature superconductors, sediment transport, turbulent flow around piers, and on and on.

When applying what you know to a new field it’s absolutely essential to do your homework before opening your mouth. It’s also important to acknowledge and respect your limitations. That’s why for the last couple of months a lot of my evening reading has consisted of journal articles about epidemiology and medicine. But knowing about numerical modeling is a lot like knowing how to read, or what to do with a wrench. Saying, “you are an astrophysicist so you don’t understand anything about epidemiological models” is a bit like saying, “you design Chevies so you don’t know anything about how Fords work.” It’s a bit like saying, “You only took a class on Agatha Christie, so you aren’t qualified to read Mickey Spillane.”

And let’s be blunt. In a world where people get their ideas about a global pandemic from an ignorant paranoid narcissistic sociopath who recommends shooting up disinfectant and shoving UV lights up your butt, scientists aren’t the ones who need to worry about overstepping their bounds.

Alas, poor CDC.

In a November 2016 essay, Autocracy: Rules for Survival, journalist Masha Gessen offered six rules for how to fight an autocrat intent on overturning a democracy. Rule #3 was, “Institutions will not save you.”

Prior to last week’s revision, the IHME models and their unreasonably low predictions for death rates were not only being brandished from the Rose Garden. They were also the models upon which the once-beyond-reproach Centers for Disease Control was basing its own projections. But by the end of the day after the IHME revised its estimates upwards, the CDC removed the IHME models from its pages altogether.

Before they were revised, the IHME models predicted approximately 60,000 COVID-19 deaths by the end of the summer. Those models were featured prominently by the CDC. After those models were revised upward from 60,000 to 135,000 deaths, the CDC dropped them altogether. Then when they were challenged by the press the CDC included them again, but now as one of many models, showing projections only out as far as June 1.

Only after their ham-handed attempt to censor the new results was called out by the press did the CDC relent. The IHME models reappeared on the page, but now buried in the middle of 14 different models. That’s OK, except that instead of showing predictions through the end of summer the CDC now only shows predictions through May. Those models, most of which assume that current social distancing practices will remain in place or even be strengthened, forecast deaths on the high side of 100,000 by the end of the month and still climbing rapidly.

But what the CDC pages absolutely do not do is point out that the modelers who had been their darlings days before are now talking about 135,000 deaths even if we remain locked down.

In the mean time, the White House stopped the CDC from releasing guidelines on how to safely reopen businesses, saying that the report “would never see the light of day.” And so it would have been were it not for someone at the CDC risking their career to pass the document to a member of the press.

A similar thing happened to the CDC guidelines for reopening schools, where detailed rules were replaced by pablum. That change escaped public attention, but followed immediately on the heals of Trump’s insistence that schools must reopen soon.

Once upon a time the CDC was the most trusted organization in the world when it came to issues of public health and disease control. Now it seems well on its way to becoming another arm of an autocrat’s propaganda machine.

“Rule #3: Institutions will not save you.”

So, what’s next?

Knowing what comes next hardly takes a rocket scientist. As the Right cries “open up!” and epidemiologists respond, “are you nuts???”, the Administration has resorted to its standard playbook. Page one reads, “hide the data, discredit the scientists, and throw shit at the walls to see what sticks.” They are already hiding the data. For example, rather than announcing vital public health data associated with outbreaks in meat packing plants, Pence is now calling those data “business matters” that are up to corporations to handle as they see fit. Discrediting the scientists… even opening that can of worms would turn this article into a tome. As for the Hail Marys, it’s China’s fault, it’s the WHO’s fault, the whole pandemic is just a ruse by hospitals to get more money, this is all a great success.

As Trump gets more and more desperate over the election I’d bet a case of good Scotch that he’s going to double down and double down again to keep people from knowing the truth of what is going on.

Then there are the double standards. As the Administration suppressed information on the explosion of COVID cases in meat packing plants, it simultaneously used the Defense Production Act to order those plants to reopen without any testing or requirements for improved safety. But when a couple of staffers at the White House tested positive (yes, unlike those workers in meat packing plants, White House staffers are tested regularly), there was immediate and thorough contact tracing and quarantines. Trump is said to have had a conniption that the virus was allowed to get that close to him personally.

I guess what’s good for the goose isn’t good enough for the gander after all.

Dr. Jeff Hester at the March for Science

Three years ago, on April 22, 2017, I went to Washington DC for the first March for Science. To be honest, that march had little effect on what was then a new Administration that has proven itself a fierce enemy of anything so inconvenient as evidence or objective truth. Today the stakes have become life or death, quite literally.

It’s time for scientists to get loud.

The past is dead. There is no going back to “normal.” But far more is at stake than our response to the COVID-19 pandemic. We are in the midst of a battle over whether truth matters. It is a battle between science and anti-science, and our lives are at stake. In the next few months that battle will decide the fate of the world in which we spend the rest of our lives and leave to our children.

The difference between a true scientist and someone who is just playing the game has little to do with the field on their diploma and everything to do with how they look at and think about the world. The defining quality of a good scientist is a first rate bullshit filter. Right now the world desperately needs people with high functioning bullshit filters who aren’t afraid to speak up.

Which brings me to my admonition for scientists. Institutions will not save you. Regardless of the discipline you call home, now is not the time for silence or circumspection. Now is not a time to hide behind academic regalia and pretend we are above such things. Now is the time to stand up for what is real, and shout loudly enough to be heard.

Reality Straight Up!

Thoughts & Observations of a Free Range Astrophysicist