- "The most pressing problem of the organization is complexity. Complexity is the primary responsibility of the enterprise architect." (Roger Sessions, Controlling Complexity in Enterprise Architecture pdf June 2007)
- "Enterprises are instances of complex adaptive systems having many interacting subcomponents whose interactions yield complex behaviors. Enterprise Architecture is a way of understanding and managing such complexity." (Beryl Bellman Managing Organizational Complexity pdf FEAC Oct 2009)
Indeed, I'm sure I've said things like this myself. But if complexity is a problem, whose problem is it? I am not seeing a huge rush of businessmen hiring enterprise architects just to deal with The Complexity Problem. Usually they have much more practical problems that they want addressing.
Complexity is a problem amplifier
So here's the thing. Apart from architects, people generally don't see complexity as their problem. What they do often acknowledge, however, is that complexity makes their problems worse. Furthermore, complexity may be one of the reasons they can't solve their problems for themselves. So complexity is a relevant factor, it's just not the problem itself.
Complexity as a lifestyle choice
And where does the complexity come from? Ironically, complexity is often created by failed attempts to reduce or eliminate complexity. In a post about the AntiFragile Enterprise (Jan 2013), Alan Hakami warns against "a tendency to build over-governed solutions that try to 'manage' complexity or uncertainty".
And where is the motivation to eliminate complexity? Suppose you go to the doctor with a headache, and the doctor says the reason you keep getting these headaches is you don't get enough fresh air and exercise. The pills just give you a temporary relief from the symptoms. You know she's right, but somehow you can't manage to get up early enough to go for a run before work. So you continue to get the headaches and you continue to need the pills. Indeed, if the pills work, you may start to exercise even less than you did before.
According to this analogy, an organization chooses the level of complexity it is comfortable with, and may well resist attempts to shift to a higher or lower level.
Complexity as a smokescreen
In my post on Devious Management and Investment Risk (January 2004), I suggested that complexity can sometimes be a deliberate tactic to conceal something, and therefore serves as a clue that something is being concealed. For example, a tangle of complicated transactions. Obviously those responsible for the concealment will resist any attempt to strip away this kind of complexity. So there is no point in attacking the complexity directly, you need to identify what is behind it.
Complexity is an opportunity amplifier
If one organization is better at handling complexity than its competitors, as a consequence of superior agility and/or intelligence, then it can use complexity as a weapon. And many organizations use this weapon against customers or regulators. As @JackGavigan says, complexity is only a problem for those who lack the brain power to deal with and exploit it.
So that gives the enterprise architect a rather different perspective on complexity, doesn't it?
Related posts: Complexity-Based Pricing (June 2008), On the Causes of Business Complexity (October 2012)
Updated 27 June 2020
Richard, I disagree in a few respects. I hope you don't mind that.
ReplyDeleteWe all know that complexity is growing rapidly today.
If we don't control complexity, the management our systems and enterprises, of their issues and their aligned evolution would increasingly require greater effort at greater costs.
Architecture helps us manage complexity though. One must document, break, group and encapsulate complexity in blocks which can be managed independently, like computer chips.
This is what the architect does and architecture comes with.
How would one be able to say what goes on within an enterprise without an architecture? How would one be able to isolate a defect in time? How would one make the strategy happen without gross gaps?
The EA architect documents the complex enterprise (otherwise described by many pictures in many drawers), establishes principles and guidelines of evolution and creates the target architecture so that complexity would grow in a controlled manner rather than feeding on itself to result in even more complexity in more diversity.
The architect also standardizes to reduce complexity.
Complexity is a problem that grows. Unmanaged, it alters the capability of the enterprise to deliver and respond to market change.
The knowledge in various domains is increasing too quickly.
How do we cope with this? People specialise. But in the process, they lose the big picture.
Hence, somebody in the enterprise has to hold this synoptic picture. This would be the EA architect.
The amount of information is growing. The Information architects then come to create and manage the information and its map, as part of EA.
As such, architects should document enterprises, guide their modular transformation, based on services, and in general should be the keepers of the big picture.
Complexity amplifies problems too, but slows down opportunity.
But Managing complexity diminishes problems and and their solving cost and amplifies opportunities.
Complexity is one of the biggest problems of an enterprise when not properly managed.
The "Unknown" comment above is mine Adrian Grigoriu. Complexity is playing its tricks here.
ReplyDeleteProving that I am not a robot when signing in is a bit too sophisticated The letters are hard to distinguish. Can you make it less complex though? You should try it yourself.
Adrian, thanks for your comment, and I agree with much of what you say. I just don't think it leads to your conclusion.
ReplyDeleteFor example, I agree that complexity alters the capability of the enterprise to deliver and respond to market change. But it also alters the capabilities of your competitors to respond to the same things. So as long as you are smarter than your competitors, complexity may tilt the playing field to your advantage.
And I agree that architects often see the need to reduce complexity, because it makes other things more difficult. But that's not generally the problem they have been presented with by the business.
What do you think would happen if the business says "We want X" and the architect says "I can't do X until I have spent three years eliminating complexity and making your enterprise more agile"?
The trick is often to deliver X in a way that reduces unnecessary complexity and increases agility - but we should never forget that this is a valuable side-effect and is not itself a direct solution to the original problem.
To make sure that we are talking about the same thing, my conclusion was that "Complexity is one of the biggest problems of an enterprise when not properly managed".
ReplyDeleteI did not propose that the architect stalls the growth in complexity, which is inevitable.
Competitors can add new functionality which comes with complexity for instance, as along as they can manage it, since otherwise it may turn against them in terms of stability, reliability, integration, response time, cost etc.
But, at similar complexity, competitors are differentiated, as well, by their capabilities to manage and not necessarily reduce this complexity.
That's how EA becomes a competitive advantage.
I also said that EA architects have the task to guide the enterprise evolution
based on an EA, principles, service orientation, roadmaps etc.
That is, if someone suggests to buy an X application of which the enterprise already as a few of the kind, the architect has to analyse and, if the proposal does not have enough business merit, suggest a solution for re-use. In fact the architect has to automate that decision for others to do, by creating upfront principles, standards... are agreed by everyone and enforced to a degree.
Thanks Adrian.
ReplyDeleteYou say "Complexity is one of the biggest problems of an enterprise when not properly managed".
I agree that anything seems to become a problem when not properly managed. So the root cause of the problem here is the lack of proper management.
And of course all the good things you say about architecture may help to restore proper management. That's how EA becomes a competitive advantage.
ReplyDelete"Everything should be made as simple as possible, but no simpler"
I'm not usually a fan of such nebulous quotes, but I think this cuts through the gap between both viewpoints; in as much as the idea of 'appropriate' (which would probably imply 'managed' or 'planned') complexity is in fact not a problem. The world is naturally complex, and every time you peel back a layer, there is only more to discover.
Architects love to make sense of these multiple dimensions of complex interactivity and relationships between things, so that they can a) understand and communicate them, and b) manipulate them (e.g. for the purpose of changing an Enterprise, etc). But in the real world, most enterprises are not appropriately complex/simple, in fact, most are a can of worms inside a bag of spaghetti, buried in a haystack; and they're doing very well if they can actually quantify that problem.
I think the semantics of the words you're using Richard engenders a common interpretation that 'complexity' means 'unnecessarily complicated, difficult and messy', whereas I think (please correct me if not) you are actually meaning 'there is a lot involved in achieving the desired outcome, which may be naturally complicated, but appropriately so for a given context'.
Too much complexity makes it difficult to change, operate or fix things efficiently; too much simplicity means it isn't actually doing what you need.
From a practical architecture perspective, I think it is a responsibility of any architecture capability to understand the complexity of a specific context, simplify the conceptual representation of the complexity so it can be worked with, and define a means by which to manipulate that complexity to effect efficient change. Heck, I would even say this is really just a facet of human psychology - understanding things through abstractions and creating 'order' is a very fundamental feature of our minds and behaviour.
We do however, have to be careful when abstracting and rationalising complexity, as everything can be conceptualised into a highly abstract model, which is congruent with the reality, but is of little use to actually solving problems that emerge from inappropriate complexity... again its about appropriateness.
So, I would say that complexity is a challenge - in as much as being able to understand and conceptualise it appropriately.
But it is also not necessarily a problem in as much as needing to remove complexity from everything everywhere as a rule.
It is more about identifying the appropriate level of complexity for the given context.. making things as simple as possible, but no simpler... :-)
Thanks for the shout out Richard.
ReplyDeleteComplexity in many cases cannot be avoided, and a certain degree of entropy can allow for innovation.
I personally believe that many architects are confusing complication with complexity. Complexity is typically in the realm of the unknown and you can have organized complexity or disorganized complexity which can be lead to chaos. Complexity makes it difficult to predict behaviors of the system over time, therefore one has to manage this effectively but not to a point where it kills productivity.
I think Adrian, if I may say so, your points are accurate when it comes to the over-engineered enterprise. Architects have to fight hard to make sure that systems/solutions are built as simple as possible.
I cannot tell you how often I see process for the sake of process, or technology for the sake of technology.
One man's simplicity is another man's complexity. Often "complexity" depends on your perspective, and something that appears simple is actually quite complex but the complexity is hidden.
ReplyDeleteThink of it as "conservation of complexity" - complexity isn't reduced or removed, it is merely moved somewhere else.
I agree with the Anonymous comment that one man's simplicity is another man's complexity, and that complexity is often merely hidden or moved around.
ReplyDeleteHowever, the term "Conservation of Complexity" suggests that the total amount of complexity remains constant. I don't think it's as simple as this. Often trying to hide or move complexity (putting complexity in the "wrong" place) has the effect of generating additional complexity.
I'm a bit late on this one... Felt between the chairs. This time, I think that the "Anonymous" was me: @enectoux.
ReplyDeleteRegarding "conservation of complexity", yes, basically this is the idea. Most of the time we are evolving in "an isolated system". E.g.: the Enterprise can be seen as an isolated system. Decreasing the complexity is either increasing it somewhere (which at the end doesn't help much) or it really decrease the other all system complexity, and then there is what we can call: "progress" or waste removal.
Even though, some of us are seeking for the 2nd alternative (removing some waste), the biggest part of the humanity is either comfortable in complexity (easy to blame) or try to give it to someone else (which is what I meant by "hide")
/Emeric
Thanks Emeric
ReplyDeleteMy experience is that organizations never do exactly what their so-called "leaders" or "designers" intend, and that there is always some emergent behaviour.
This behaviour may seem unnecessarily complicated or complex, but this is caused by events (or its interpretation of these events). A complex system "evolves" to achieve a locally efficient response to its perceived environment.
Even if we regard an enterprise as an isolated system, as Emeric suggests, clumsy attempts to reduce this complexity are often counterproductive, merely shifting the system from its local optimum, and therefore making things worse rather than better. This is akin to what Deming calls tampering or meddling.
Hence my view that the problem is usually not complexity as such but the failed attempts to deal with complexity.
Emeric talks about people or organizations being comfortable with complexity. This is one of the benefits of (organizational) intelligence.