Fall of the Experts

In July of 2020, I was heartened by an interview by Freddie Sayers on Unherd with Anders Tegnell, the architect of Sweden’s COVID response.

The interview was full of nuanced and common-sense statements by Tegnell. For example, he pointed out the lack of evidence and precedence for draconian lockdowns and their potential for enormous collateral harm:

“Of course we are trying to keep the mortality rates as low as possible, but at the same time we have to look at the draconian measures you are talking about. Are they going to produce even more deaths by other means than the disease itself? Somehow we need to have the discussion of what we are actually trying to achieve. Is it better for public health as a whole? Or is it trying to suppress Covid-19 as much as possible? Because getting rid of it I don’t think is going to happen: it happened for a short time in New Zealand and maybe Iceland and those kind of countries might be able to keep it away, but with the global world we have today, keeping a disease like this away has never been possible in the past and it would be even more surprising if it were possible in the future.”

Even more impressive was Tegnell’s humility. Several times during the interview he said “we don’t know,” and he qualified many of his answers with uncertain terms such as “seems” and “might.” I thought that was exactly what experts should have been doing all along, communicating nuance and even uncertainty to a terrified public. Either that wasn’t happening at all, or the media was filtering out all the nuance and uncertainty any expert might offer and just went with certain doom.

I texted a link to the interview to my sister, who I describe in my book Fear of a Microbial Planet as a germophobe. She was obviously worried about contracting the virus early on, but recently had been showing some healthy skepticism about the doom and gloom she was seeing on the news. Interestingly, she responded with “The only thing I don’t like, but it is the truth, is that he keeps saying ‘we don’t know.’ That is what scares me, is the ‘don’t know’ part of any of it.” The humility and uncertainty on display in the interview had given me comfort, but for my sister, it had the opposite effect.

The more I thought about it, the more I realized that I was the outlier. Most people don’t want nuance and uncertainty when they are scared. They want to know that there are experts that know everything that is going to happen and how to stop it. They want to know that all risk of disease and death can be eliminated with simple and sustainable countermeasures, and they are quite willing to trade away many of their freedoms, even for an illusion of control. Many experts and the media that promote them are perfectly happy to sell that illusion when the public is frantically buying.

Because experts failed so miserably to live up to the public and media’s magical thinking the last three years, the word “expert” has lost a lot of its meaning, and that’s not necessarily a bad thing. Experts are terrible at predictions and don’t have much knowledge outside of their often narrow fields of interest. In a very complex situation such as a pandemic, there will not be any one person who has a deep understanding of what’s happening at any given moment, much less the ability to predict what will happen next. It’s like asking the CEO of a car manufacturer to build a car by himself from scratch—it’s nearly impossible because it requires the coordinated efforts of hundreds of people specializing in the construction of each part and assembly of the finished product. Not even a CEO could perform each step.

In Chapter 11 in my book, I explain why experts aren’t very good at predictions and don’t have as much knowledge outside of their fields as we expect of them:

In the early days of the pandemic, the amount of coronavirus “experts” was limited, and there was a lot of competition for the few that might have qualified in media circles. One of the unquestioned experts was my former PhD advisor, Dr. Stanley Perlman, a coronavirologist/immunologist at the University of Iowa. Stan had been thrust into the world of human coronavirus research after the SARS1 outbreak put the spotlight unexpectedly on human coronaviruses. He had helped start a BSL3 lab at Iowa and began working on SARS1 infection in mice, while also paying attention to other coronaviruses with potential to cause serious disease, like the Middle East Respiratory Virus, or MERS.

When only two cases of SARS-CoV-2 infection had been confirmed in the United States, an Iowa TV station sought out Stan for a prediction about how the U.S. would be affected by the novel virus. People were already seeing horror stories from China, which had just locked down the day before. They wanted some reassurance. Thinking about how SARS1 had been contained over the course of several months in 2003, Stan told the reporter he thought Iowa would never see a case. Obviously, that prediction didn’t age well.

Two years later, when I asked him about his early recollections, he brought up that interview, “The biggest mistake I made in my initial impression is that the number of cases was increasing but I thought it was still consistent with a SARS and MERS-like spread, whereas mostly lower respiratory tract. So, in the beginning I thought that this was going to be like SARS1 and MERS and that quarantining will work. And within five weeks we knew that wasn’t going to work. When you’re asked that question as an expert you really have to walk the line and not being really sure where you are with two cases, do you say, “Well, I think we all have to be really worried because it seems to be spreading quickly,” when there really wasn’t that much evidence for that or do you say, “Well, it’s only two cases.” And I opted for saying “It’s only two cases, and I think we should just see how it plays out.”” Not only were most people clueless about how SARS-CoV-2 would behave, experts like Stan didn’t know either. His expertise was actually problematic at such an early time point.

Experts are generally terrible at forecasting, as demonstrated by psychologist and author Philip Tetlock in his 2005 book Expert Political Judgement. In Tetlock’s study, when 284 experts were asked to make 27,451 predictions in areas relevant to their expertise, the results were a total bust. When pitted against “dilettantes, dart-throwing chimps, and assorted extrapolation algorithms,” experts did not consistently perform better than any of them. They were no more accurate at forecasting than the average person. However, there were some people who proved better at forecasting, yet these were not what one would traditionally label as “experts.” Instead, more accurate forecasters tended to be more well-rounded, less ideological, and more willing to challenge their own assumptions. In contrast, experts just assumed they knew everything, and were wrong as much as right.

The wildly inaccurate predictions of many experts and pandemic prediction models only confirmed Tetlock’s conclusions. Experts were repeatedly wrong in every direction. Infectious disease epidemiologist John Ioannidis, one of the most cited scientists of all time, told CNN personality Fareed Zakaria in April of 2020, ‘‘If I were to make an informed estimate based on the limited testing data we have, I would say that COVID-19 will result in fewer than 40,000 deaths this season in the USA.’’ By June 18, 2020, the estimated number of U.S. deaths from COVID-19 was 450,000. Nobel Laureate and Stanford professor Michael Levitt developed models he used to claim that the virus was already peaking in late March of 2020. At the end of July, Levitt predicted that the pandemic would be over in the U.S. by the end of August, with less than 170,000 deaths. Instead, the number was around 180,000 by the end of August, and steadily climbing.

And that was just the COVID “minimizers.” Many COVID “maximizers” were just as wrong, yet they were the ones that leaders were heeding. On March 27th, 2020, Dr. Ezekiel Emanuel, chair of the department of medical ethics at the University of Pennsylvania, predicted 100 million cases of COVID-19 in the U.S. in just four weeks. Four weeks later, on April 27th, 2020, there were one million confirmed cases. The infamous Imperial College Model, developed by Professor Neil Ferguson and colleagues, predicted over 2 million U.S. deaths within three months of the beginning of the pandemic. This was an enormously influential model, as White House Coronavirus Response Coordinator Deborah Birx admitted it was used to promote nationwide shutdowns in her 2022 book Silent Invasion.

Instead of a complete collapse of the U.S. healthcare system, three months later in June there were ~109,000 deaths. The equally influential IHME models predicted a massive, overwhelming surge in patients requiring hospital beds and ventilators. New York Governor Andrew Cuomo said on March 24th that the state could need up to 140,000 hospital beds (out of an available 53,000), with 40,000 ICU beds needed. Just two weeks later, with cases rapidly decreasing, only 18,569 hospitalizations had been reported. Although several hospitals had reached or exceeded capacity during the surges in New York and New Jersey, many remained nearly empty, with some even laying off staff. Two months later, after it was clear the predicted surge wasn’t going to materialize, Cuomo admitted the information he received from the experts was terrible, “All the early national experts. Here’s my projection model. Here’s my projection model. They were all wrong. They were all wrong.”

Once U.S. states began to reopen, models again wrongly predicted massive COVID resurgence. Georgia’s reopening was criticized in the press as an “Experiment in Human Sacrifice.” A model developed by researchers at Massachusetts General Hospital in Boston predicted that even a gradual lifting of restrictions on the planned date of April 27th would result in over 23,000 deaths, while keeping current restrictions until July would result in ~2,000 deaths. Keeping restrictions wasn’t what the modelers recommended, as additional results showed a stricter 4-week lockdown would have the best outcome.

None of that even remotely happened. One month after Georgia reopened, instead of 23,000 deaths, 896 were recorded. Georgia was not an isolated example. All over the U.S., states that reopened were predicted to have surges in cases that rarely materialized in the predicted time frame. “Just wait two weeks, and you’ll see,” maximizers would say, ad nauseum. When two weeks and more passed, maximizers would explain the discrepancy by pointing out that the apocalyptic forecasts were made to show what would happen if there were no lockdowns, restrictions, or mandates. The outcome could be therefore easily explained by “It could have been so much worse without government action.”

There was a huge, glaring problem the maximizers had to ignore to make that argument, rooted in the fact that not every country or state responded to the pandemic threat with lockdowns and mandates. Sweden did not lock down or close primary schools—forced mitigation measures were limited to gatherings of over 50 people and others were mostly voluntary, with the government emphasizing personal responsibility over coercion. When a team of Swedish researchers applied the Imperial College model to Sweden, the output predicted ~96,000 deaths for unmitigated spread. Imperial’s own numbers for Sweden came very close, topping out at over 90,000 deaths. Even with lockdowns and other forced mitigation measures, more than half that number were still predicted by the model, with 40-42,000 deaths. Yet in response to the modest restrictions that were instituted, the virus refused to follow maximizer models, and Sweden instead suffered 13,000 COVID deaths in the first year of the pandemic. This was less than half of what was projected, even with full-on-Imperial-College-style lockdowns, much less than what was projected if they did nothing at all.

In hindsight, it’s very clear that numbers aren’t substitutes for arguments, yet that’s exactly how predictions were viewed early on in the pandemic. For maximizers, cataclysmic predictions generated by models and experts served to promote lockdowns, mandates, and behavioral changes—they scared the crap out of people and made them stay home and away from others. It simply didn’t matter if the predictions were correct, the ends were justified by the means. For minimizers, large numbers only increased the potential for collateral damage, because they knew the bigger the numbers, the more draconian restrictions would be accepted. Thus, less catastrophizing would result in less hasty and damaging decisions by leaders. Ultimately, both groups were both right and wrong. COVID mortality was high in the United States, with over a million recorded deaths, but it happened over the course of two years and through several waves which few predicted.

Rather than arguing about numbers, the main arguments should have focused on what could be done to minimize the damage of a global pandemic without causing more collateral damage. The arguments were one-sided—maximizers won in many places, not through debates about evidence, but by attacking and censoring their opposition and by selling illusions of control and consensus to a frightened public.

The pandemic opened the curtain to expose the folly of expert worship. Experts are just as fallible and prone to biases, toxic groupthink and political influence as anyone else. This recognition might make people uneasy. However, it should also force a sense of responsibility to search for the truth despite what the experts might say, and that’s a good thing.