A corollary of this theory that is of particular interest to architects and complex system engineers concerns the design of fail-safe mechanisms. Nuclear power and oil extraction are examples of environmentally critical operations; they are therefore subject to detailed risk assessment, and designed with multiple fail-safe mechanisms. And yet both the oil spillage last year in the Gulf of Mexico and the partial melt-down in Japanese nuclear reactors following the recent tsunami involved the simultaneous failure of multiple fail-safe mechanisms. Obviously that's not supposed to happen.
Simultaneous failure of supposedly independent mechanisms is a Black Swan event.
Update (August 2011)
A recent study by Oxford University and McKinsey has blamed rare but high-impact problems, dubbed "black swans", for the increasingly common phenomenon of large IT project whose cost spirals out of control. The study finds this phenomenon to be three times as common in IT than in other domains [BBC News, 26 August 2011]. See my post on Black Swan Blindness.
Update (October 2011)
Reviewing a couple of recent books about BP and the oil spill in the Gulf of Mexico, Mattathias Schwartz makes a number of relevant points.
When crucial pieces of our infrastructure fail, they do so gracelessly, without much warning and in ways that are difficult to anticipate. ... The failure to grasp the possibility of system-wide failure might be one in an accelerating series, bookended by the 2008 financial crisis and the Fukushima nuclear meltdown last spring.
One reason for the oil and gas industry’s quick comeback in the US was the successful packaging of the blowout as a ‘black swan’, an event of such low probability that it couldn’t have been anticipated. This certainly helped excuse the fact that no one – not BP, Chevron, Exxon or Shell – had a working plan for plugging a blowout as deep as Macondo .
BP ... claimed, in its own report on the blowout, that the event had eight causes, of which BP was partly responsible for one. The president’s commission concluded that the disaster had nine causes, and that BP was responsible for six or seven. And yet BP stands by what it said at the start.
The size of the system and the complexity of the data make it possible to argue for a maddeningly wide range of positions, especially when it comes to vague legal notions like ‘negligence’ or ‘responsibility’. Both concepts hinge on proving that one linear narrative is the right one.
Mattathias Schwartz, LRB 6 October 2011
reviewing
- Spills and Spin: The Inside Story of BP by Tom Bergin
- A Hole at the Bottom of the Sea: The Race to Kill the BP Oil Gusher by Joel Achenbach
