COVID-19 - Anarchy or Panarchy?

In September 2005, we had reason to worry about the ability of a tightly coupled world to withstand shocks. At that time this included Hurricane Katrina and SARS. More recent crises, including the COVID-19 pandemic and the war in Ukraine have arguably outshocked these.

In his analysis of the economic sanctions imposed against Russia following its 2022 invasion of Ukraine, Simon Jenkins comments that the interdependence of the world’s economies, so long seen as an instrument of peace, has been made a weapon of war.

As the global economy becomes more tightly coupled, the chances of one event having a catastrophic impact on the entire system increase notes an account called @Forrest. Forrest, billed as anti-tech, right-wing dissident thought, uses this statement as part of an argument against geoengineering remedies to climate change, on two grounds. Firstly, because meddling with complex systems is likely to have unforeseen consequences, and secondly because these supposed remedies represent a further power-shift towards global technological elites. (Bill Gates obviously, who else?)

Other thinkers see this as an opportunity to shift from deterministic systems to more adaptive and resilient systems (Wieland) or to shift from technological capitalism to a different sociopolitical system (Zhang).

 

---

Tony Dutzik, Defusing a rigged to blow economy: Rebuilding resilience in a suddenly fragile world (Frontier Group, 30 March 2020) reprinted (Strong Towns, 1 April 2020) 

Nick Gall, Panarchitecture: Architecting a Network of Resilient Renewal (Gartner, 24 January 2011)

Tim Harford, Why the crisis is a test of our capacity to adapt (Financial Times, 20 March 2020)

Simon Jenkins, The rouble is soaring and Putin is stronger than ever - our sanctions have backfired (The Guardian, 29 July 2022)

Andreas Wieland, Dancing the Supply Chain: Toward Transformative Supply Chain Management (The Journal of Supply Chain Management. 2021 Jan; 57(1): 58–73.  

Yanzhu Zhang, Is panarchy relevant in the COVID-19 pandemic times? (Blavatnik School of Government, 10 June 2020)

Related blogposts: Efficiency and Robustness - On Tight Coupling (September 2005)

Updated 18 February 2023

Testing Multiples

From engineering to medicine, professionals are often forced to rely on tests that are not always completely accurate. In this post, I shall look at the tests that millions of people were obliged to use during the pandemic, to check whether they had COVID-19. The two most common tests were Lateral Flow and PCR. Lateral Flow was quicker and more convenient, while PCR took longer (because the sample had to be sent to a lab) and was supposedly more accurate.

There was also a difference in the data collected from these tests. Whereas all the results from the PCR tests should have been available in the labs, the results from the lateral flow tests were only reported under certain circumstances. There was no obligation to report a negative test unless you needed access to something, and people sometimes chose not to report positive tests because of the restrictions that might follow. And of course people only took the tests when they had to, or wanted to. When people had to pay for the tests, this obviously made a big difference.

To compensate for these limitations, some random screening was carried out, which was designed to produce more reliable and representative datasets. However, these datasets were much smaller.

 

So what can we do with this kind of data? Firstly, it tells us something about the disease - whether it is distributed evenly across the country or concentrated in certain places, how quickly it is spreading. If we can combine the test results with other information about the test subjects, we may be able to get some demographic information - for example, how is the disease affecting people of different age, gender or race, how is it affecting different job categories. And if we have information from the health service, we can estimate how many of those testing positive end up in hospital.

This kind of information allows us to make predictions - for example, future demand for hospital beds, possible shortages of key workers. It also allows us to assess the effects of various protective measures - for example, to what extent does mask-wearing, social distancing and working from home reduce the rate of transmission.

Besides telling us about the disease, the data should also be able to tell us something about the tests. And the accuracy of the predictions provides a feedback loop, which may enable us to reassess either the test data or the predictive models.

 

In her book The Body Multiple, Annemarie Mol discusses the differences between two alternative tests for atherosclerosis, and describes how clinicians deal with cases where the two tests appear to provide conflicting results, as well as cases where there may be other reasons to question the test results. Instead of having a single view of the disease, she talks about its multiplicity or manyfoldedness.

But questioning the test results in a particular case, or highlighting particular issues with a given test, does not mean denying the overall value of the test. Most of the time we can continue to regard a test as useful, even as we are considering ways of improving it.

If and when we introduce a new or improved test, we may then wish to translate data between tests. In other words, if test A produced result X, then we would have expected test B to produce result Y. While this kind of translation may be useful for statistical purposes, we need to be careful about its use in individual cases.

For many people, the second discourse appears to undermine the first discourse. If we can't always trust the data, can we ever trust the data? During the COVID pandemic, many rival interpretations of the data emerged; some people chose interpretations that confirmed their preconceptions, while others turned away from any kind of data-driven reasoning.

 

The COVID pandemic became a politically contentious field, so what if we look at other kinds of testing? In safety engineering, components and whole products are subjected to a range of tests, which assess the risk of certain kinds of failure. Obviously there are manufacturers and service providers with a commercial interest in how (and by whom) these tests are carried out, and there may be regulators and researchers looking at how these tests can be improved, or to detect various forms of cheating, but ordinary consumers don't generally spend hours on YouTube complaining about their accuracy and validity.

Meanwhile even basic corporate reporting may be subject to this kind of multiplicity, as illustrated in my recent post on Data Estimation (July 2022).

So there is a level of complexity here, which not all data users may feel comfortable with, but which data professionals may not feel comfortable about hiding. In a traditional report, these details are often pushed into footnotes, and in an online dashboard there may be symbols inviting the user to drill down for further detail. But is that good enough?

---

Annemarie Mol, The Body Multiple: Ontology in Medical Practice (Duke University Press 2002)

Wikipedia: COVID-19 testing

Related posts: Data-Driven Reasoning - COVID (April 2022), Data Estimation (July 2022)

Data-Driven Reasoning (COVID)

Following the data is all very well, but how should we decide which data to follow?

 

Nate Silver argues that the total data are more important than the marginal data. There is more virus transmission in restaurants than in aeroplanes.

The average American spends something like 5 hours per year on a plane. The mask mandate might be good or bad at the margin, but it is very unlikely to make a major difference in the overall course of the coronavirus.

— Nate Silver (@NateSilver538) April 19, 2022

The average American eats at a restaurant about 250 times per year, whereas they travel by airplane about 2.5 times per year. Dining out and general nightly social activity is a much, much, much bigger contributor to COVID spread than air travel.

— Nate Silver (@NateSilver538) April 19, 2022

 

Several people have challenged this, including Carl Bergstrom.

Imagine thinking that this is what data-driven reasoning looks like 

— Carl T. Bergstrom (@CT_Bergstrom) April 19, 2022

The first point I want to make is about risk. When calculating the risk of transmission in aeroplanes, the level of risk somewhere else is not really relevant. As Bergstrom points out, the calculation should be based simply on the costs and benefits of wearing masks in that particular setting.

To explain: whether or not to take some safety precaution requires a cost-benefit analysis.

Nate forgot the cost side of the equation.

Wingsuit jumping be far less common than motorcycle riding, but that's hardly an argument against safety precautions for squirrel-suiters.

— Carl T. Bergstrom (@CT_Bergstrom) April 19, 2022

The problem here, of course, is determining the cost of wearing masks. If some people regard mask-wearing as a minor inconvenience, while others regard it as a major infringement of their human rights, what sort of data would be relevant to this calculation? Or if mask-wearing impedes communication to some extent, can we quantify this effect?

The broader question is whether we should be looking at total costs and benefits, or marginal costs and benefits? In the case of mask-wearing, if we assume that most people possess a usable mask, then the marginal cost of wearing one might simply include the increased frequency of washing or replacing it, plus whatever psychological, physiological or social costs we can determine. 

But for some people, opposition to mask-wearing is much more fundamental than that. It is not about the costs and benefits of mask-wearing in a particular setting, but an overall objection to living in the kind of society where mask-wearing is mandated at all. Some people even object (violently) to the sight of other people wearing masks. So any kind of mask-wearing may cause social division, therefore incurring a sociopolitical cost, and this needs to be set against the overall benefits of controlling the transmission of disease.

Nate Silver acknowledges that a mask-wearing mandate may be useful at the margin, but questions its overall value. Presumably people aren't going to wear masks in restaurants while they are eating. So his tweet seems to be asking what is the point of making them wear masks on planes?

This then brings us onto a question about policy, and the extent to which policy can be evidence-based. Is it better to have a consistent policy on mask-wearing across different settings, or to focus policy on those areas that are regarded as the highest risk? And should policies be optimized for effectiveness (what will have the greatest effect on suppressing transmission of the virus) or social acceptability (what will most people accept as reasonable)? Nate Silver's data may be relevant to this calculation, at least to the extent that they influence people's opinions.

While there are some particularly emotive features of this example, it raises some important general points about data-driven reasoning, especially in relation to cost-benefit calculations and evidence-based policy.

Business Science and its Enemies

#FollowingTheScience As politicians around the world struggle to contain and master the Covid-19 pandemic, the complex role of science in guiding decision and policy has been brought into view. Not only the potential tension between science and policy, but also the tension between different branches of science. (For example, medical science versus behavioural science.)

In this post, I want to look at the role of science in guiding business decisions and policies. Professor Donoho traces the idea of data science back to a paper by John Tukey in the early 1960s, and the idea of management science, which Stafford Beer described as the business use of operations research is at least as old as that. More recently, people have started talking about business science. These sciences are all described as interdisciplinary.

Operations research itself is even older. It was originally established during the second world war as a multi-disciplinary exercise, perhaps similar to what is now being called business science, but it lost its way in the years after the war and was eventually reduced to a set of computer programming techniques with no real impact on organization and culture. 

In a recent webinar on Business Science, Steve Fox asked what business science enabled leaders to do better, and identified three main areas. 

Firstly system-related - to anticipate requirements and resources, identify issues, including risk and compliance issues, and fix problems. 

And secondly people-related - to tell the story, influence stakeholders and negotiate improvements. Focusing on message and communications to the various audiences we need to influence is a key part of business science. 

And thirdly, thinking-related. When business science is applied correctly, it changes the way we think. 

I agree with these three, but I'd like to add a fourth: organizational learning and agility. This is an essential component of my approach to organizational intelligence, which is based on the premise that serious business challenges require a combination of human/social intelligence and machine intelligence.

Steve Fox also stated that the biggest obstacles to creating data-driven business aren't technical; they're cultural and behavioural. So in this post, I also want to look at some of the obstacles of following the science in the context of business and organizational management.

• Poor Data - Inadequate Measurement and Testing - Ignoring Critical Signals
• Too Much Data - Overreliance on Technology - Abdication
• Silo Culture - Someone Else's Problem
  
• Linear Thinking - Denial of Complexity
  
• Premature Attempts to Eliminate Uncertainty
• Quantity becomes Quality

After I had initially drawn up this list, I went back to Tukey's original paper and found many of them clearly stated in there. 

---

Poor Data - Inadequate Measurement and Testing - Ignoring Critical Signals

Empirical science relies heavily on a combination of observation, experiment and measurement. 

Too Much Data - Overreliance on Technology - Abdication

Tukey: Danger only comes from mathematical optimizing when the results are taken too seriously.

Adrian Chiles reminds us that all the data in the world is no use to you if don’t know what to do with it. He quotes Aron F. Sørensen (via Chris Baraniuk) Maybe today there’s a bit of a fixation on instruments.

And in many situations, people overrely on algorithms. For example, judges relying on algorithms to decide probation or sentencing, without applying any of their own judgement or common sense. If a judge doesn't bother doing any actual judging, and lets the algorithm do all the work, what exactly are we paying them for?

Linear Thinking - Denial of Complexity

Tukey: If it were generally understood that the great virtue of neatness was that it made it easier to make things complex again, there would be little to say against a desire for neatness.

One of the best-known examples of linear thinking was a false theory about the vulnerabilities of aircraft during the second world war, based on the location of holes in planes that returned to base. People assumed that the vulnerabilities were where the holes were, and this led to efforts to reinforce planes at those points.

Non-linear thinking turns this theory on its head. If a plane makes it back to base with a hole at a particular location, this should be taken as evidence that the plane was NOT vulnerable at that point. What you really want to know is the location of the holes in the planes that did NOT make it back to base.

In 1979, C West Churchman wrote a book called The Systems Approach and its Enemies, about how people and organizations resist the ways of thinking that Churchman and others were championing. Among other things, he noted the way people often preferred to latch onto simplistic one-dimensional/linear solutions rather than thinking holistically.

---

Chris Baraniuk, Why it’s not surprising that ship collisions still happen (BBC News 22nd August 2017)

Christa Case Bryant and Story Hinckley, In a polarized world, what does follow the science mean? (Christian Science Monitor, 12 August 2020)

Adrian Chiles, In a data-obsessed world, the power of observation must not be forgotten (The Guardian, 5 November 2020)

C West Churchman, The Systems Approach and its Enemies (1979)

David Donoho, 50 years of Data Science (18 September 2015)

John Dupré, Following the science in the COVID-19 pandemic (Nuffield Council of Bioethics, 29 April 2020)

Faye Flam, Follow the Science Isn’t a Covid-19 Strategy: Policy makers can follow the same facts to different conclusions (Bloomberg, 10 September 2020)

Steve Fox, A better framework is needed: From Data Science to Business Science (Consider.Biz, 17 September 2020) via YouTube

Matt Mathers, Ministers using following the science defence to justify decision-making during pandemic, says Prof Brian Cox (Independent, 19 May 2020) 

Megan Rosen, Fighting the COVID-19 Pandemic Through Testing (Howard Hughes Medical Institute, 18 June 2020)

John Tukey, The future of data analysis (Annals of Mathematical Statistics, 33:1, 1962)

Wikipedia: Data Science, Management Science 

Related posts: Enemies of Intelligence (May 2010), Changing how we think (May 2010), Data-Driven Reasoning - COVID (April 2022)