Abstract
During the Covid-19 pandemic, the best-performing modelling groups were not always the best-resourced. This paper seeks to understand and learn from notable predictions in two reports by the UK's Scientific Advisory Group for Emergencies (SAGE). In July 2021, SAGE reported that, after the upcoming lifting of restrictions (“Freedom Day”) cases would “almost certainly remain extremely high for the rest of the summer” and that hospitalisations per day would peak between 100 and 10,000. Cases were not “extremely high” and began to decline, while hospitalisations initially lay outside (above) SAGE's confidence bounds, and only came within the expected range when the upper and lower bound moved so far apart as no longer to be useful for policy or planning purposes. The second episode occurred in December 2021, when SAGE projected 600–6000 deaths per day at peak in the scenario where restrictions remained as they were (referred to as “Plan B"). In the event, restrictions did not change, and deaths peaked at 202, well below the lower bound, even though this spanned one order of magnitude. We argue that the fundamental problem was over-reliance on mechanistic approaches to disease modelling, and that a methodologically pluralist approach would have helped. We consider various ways this could have been done, including evaluating past performance and considering data from elsewhere. We show how the South African Covid-19 Modelling Consortium performed better by learning from experience and using multiple methods. We conclude in favour of methodological pluralism in infectious disease modelling, echoing calls for methodological pluralism in recent literature on causal inference.