(In my blogpost on The Future of Enterprise Architecture, I had asserted that real business is inherently complex. Lawrence's question can be found in the comments to this blogpost.)
What is complexity?
There are several competing notions of complexity in the literature, and circulating on the Internet, and I don't want to get bogged down in complexity theory. So I'm going to see how far I can get with this discussion without a formal definition of complexity. Instead of a definition, I am going to list some of the characteristic features of complex systems, taken from a book by Flood and Carson, derived in part from a paper by F.E. Yates.
1. Significant interactions
2. High number of parts, degrees of freedom or interactions
3. Non-linearity
4. Broken symmetry / asymmetry
5. Non-holonomic constraints - for example the Falling Cat Problem
6. Hierarchy and emergence
7. Aesthetics
R.L Flood and E.R. Carson, Dealing with Complexity (1988)
Causes of complexity
There are several possible causes of complexity and/or complication in systems. Roughly speaking, we can identify at least
- Emergent complexity - complexity that is the consequence of many small and apparently unrelated decision and actions, which interact in unanticipated ways. Complicated IT systems can often be explained in this way, and some IT experts talk as if all complexity can be explained thus.
- Transitional complexity - complexity that is caused by the need to maintain multiple operational models during transformation, especially when there are many different transformations underway at the same time. And when large organizations such as the NHS are undergoing constant and continual change, there may be no end to this form of complexity. (I am indebted to Richard Gilyead for adding this one - see comments below.)
- Perverse complexity - complexity that is caused by clumsy efforts to reduce complexity. Often the complexity is merely displaced or exported into a system that is even less able to cope with it, thus weakening the overall ecosystem.
- Contrived complexity - complexity that is deliberately created to benefit some stakeholders (vested interests) at the expense of others. This is sometimes called Complexity-as-a-Weapon. See my post on Complexity-Based Pricing.
- Regulatory complexity - see my post on Regulation and Complexity.
- Irreducible complexity - complexity that reflects the real complexity of the demand environment.
In some circumstances, these factors may combine to produce yet more complexity.
Examples of irreducible business complexity
As I see it, there are many types of irreducible complexity in business, including various forms of asymmetry, indirect value, multi-sidedness and non-linearity.
For example, a supermarket chain can optimize/innovate its relationships with its suppliers, or it can optimize/innovate its relationships with its customers, or it can optimize/innovate its store layout. Complexity arises when it tries to optimize and innovate all three simultaneously - which is of course exactly what it does need to do in order to keep one step ahead of its competitors.
For example, complexity arises in the health service when it tries to orchestrate thousands of different clinical pathways, using any number of expensive resources, in any number of permutations, in a way that delivers both a joined-up experience for the patient and a cost-effective system for the taxpayer.
These kinds of complexity are not caused by support services such as IT or HR but by the structure of demand in these sectors. One challenge for IT is to build appropriate information systems and platforms that support the effective management of real business complexity. (One of the problems with the NHS National Programme for IT (NPfIT) was that despite having cost a vast amount of money, the system simply didn't have the requisite variety to tackle the real problems, and was seen as an irrelevant white elephant by a critical mass of influential stakeholders.) And one challenge for HR or OD is to build appropriate organization structures and incentive schemes that align with the complexities of the business.
Final remarks
I have sometimes talked about irreducible complexity in terms of requisite variety - the idea that an enterprise needs to be aligned with the complexity of the environment. However, this term may be misapplied, as it focuses on variety as being the most important aspect of complexity - but in other situations, we may wish to look at other aspect of complexity, such as Asymmetric Demand.
In any case, the alignment of the complexity of the enterprise with the complexity of the environment is a strategic choice. An enterprise may choose to ignore or attenuate external complexity, and offer a simplistic one-size-fits-all solution to its customers. Many large organizations do exactly this, possibly destroying long-term value for themselves and their customers, and yet they roll on from crisis to crisis, until some crisis finally finishes them off.
The capability of an organization to manage a given level of complexity is linked to what I call Organizational Intelligence. I believe that increasing organizational intelligence has an indirect strategic value, because it increases the ability of the organization to align itself with its environment. We may frame the benefits of this in terms of sustainability and business excellence.
Afterword: Tackling complexity
Some comments here and on Twitter remind me about various approaches that may be used for tackling particular forms of complexity, including Continuous Modernization, Simple Iterative Partitions (SIP) and the Theory of Constraints (TOC). Typically these are focused on a specific problem, such as better ROI from IT investments (@RSessions). These approaches may be successful in some situations, but the notion of complexity outlined here is broader than the range of any popular complexity-busting tool.
(updated March 20th 2013)
Thank you for sharing. I did not categorize it so far as nice as you do here but this is easily be usable to trace down (or hunt down) root causes of complexity and use that in argumentation chains.
ReplyDeleteJust perfect.
Continuing your theme of "causes of Complexity" you might be interested in the Continuous Modernization pattern that I have recently published. See Solution section.
ReplyDeletehttp://www.cbdiforum.com/evolutionary-delivery
Thanks David. I am happy to see that the CBDI Forum's current position on Evolutionary Delivery is a practical refinement of some of the ideas of Organic Planning we were working on together a few years ago.
ReplyDeleteI believe that there is another type of complexity: Change Complexity. This is the result of having to maintain multiple operating models during a transition period. Often, there is a multiplying effect where complexity is increased before a simpler, more rational model can be implemented. Unfortunately, some organisations suffer continual change, thus ensuring complexity is never reduced (e.g. NHS).
ReplyDeleteGood stuff, Richard. There are many aspects to the idea of complexity. The Theory of Constraints community tries to argue that a given business is not complex, if you can find the right point of view. There should be one leverage point by which you can pull along the entire business, rather than many that must be pulled simultaneously to make the business grow.
ReplyDeleteHowever, humans are involved. And we seem to create more complexity than is necessary. We like to separate the whole into conveniently-sized pieces. And then we assume that operating each piece independently well will create a well-running whole. This rarely works. Then there is the fear of uncertainty, which leads us to break things down as well and attempt to model them to within an inch of their lives. (Forecasts are a beautiful example: we forecast in an attempt to create order on today. But forecasts are by their very nature wrong. Why not figure out a way to operate that responds much faster to reality, instead of the imaginings of a forecast?) And then there is the funny human fear of conflict, and the things we do individually and as groups to avoid it: compromise. We go for the compromise solution, rather than dealing with the conflict head on. The compromise adds another layer of processes to the system. And that layer is often unspoken, so we get unwritten rules and "the way we do things around here" embedded in everything. Making it harder to resolve the next time we come around to it.
Thanks Jack. Lots of valid observations here, but I am not convinced either (a) that all forms of complexity can be explained by the various phenomena you mention, or (b) that all forms of complexity can be handled by the Theory of Constraints. You say there should be one leverage point, but why must there be?
ReplyDeleteOh, and belated thanks to RichardG for his very useful comment about Change Complexity, which I have already added to the main post. (Calling it Transitional Complexity, hope you don't mind.)
ReplyDeleteI should clarify. The points I posted were ways we tend to make things more complex/complicated, not the only reasons.
ReplyDeleteAnd you are right that not all forms of complexity fit the TOC mindset. Rather, TOC comes to system problems from the perspective that there is most likely a core problem (core conflict) that is connected to most of the current problems that the system creates. If you can find this core and a way to resolve it, then you should also resolve all the bad effects that result. (And if they don't resolve, then you haven't found the core problem.)