Wednesday, September 15, 2010

The Future of Enterprise Architecture

Let's start with the positive.

Enterprise architects are mostly pretty skilled at identifying duplication, inconsistency and lack of interoperability across complex systems of systems, which they regard as fragmentation and waste. As a consequence, they typically often set themselves a mission of simplification and unification. For some, this mission is largely confined to IT systems; for others, this mission may extend to business processes and the entire business operation.

In pursuing this mission, they find themselves opposed to narrow or short-term thinking from various stakeholders - line managers wanting quick and cheap solutions to local problems, project managers or external contractors trying to meet tight project goals, infrastructure managers wanting to protect established assets and arrangements. A committed enterprise architect will therefore get into a series of battles inside the organization, defending the principles of simplification and unification. (An uncommitted enterprise architect will withdraw from these battles and devote energy to producing abstract plans that will be ignored by everyone else - but it's difficult to see any value whatsoever in this kind of disengagement.)

The primary weapon in these battles is the Plan - an abstract blueprint that represents an idealized TO-BE architecture. This Plan needs to embody the principles of simplification and unification, not only because that's the mission, but also because that's the only way it's going to make sense to senior management, without whose support the battles cannot be won. The Plan is therefore not just a weapon but a story, perhaps even a myth.

But here's the problem: real business is inherently complex. Therefore battles against complexity (not always, but often enough to cause concern) are ultimately battles against the business. Or worse - against the customers of the business.

The mission to simplify and unify has undoubtedly had some successes, but it has also produced some massive failures - projects that have gone way over budget without coming anywhere close to delivering the promised business benefits. Some of the best-known examples can be found in the public sector because these are open to public scrutiny, but the private sector is not immune to this kind of failure. In the UK this week, the National Programme for IT in the health service has been finally put out of its misery, following cancellation of a number of other projects that were supposed to deliver both improved outcomes and massive cost savings.

To the extent that projects like this appear to be following an old-style enterprise architecture agenda, these failures serve to undermine the credibility of enterprise architecture in the eyes of many stakeholders, and should pose a series of hard questions for enterprise architecture practice.

Of course enterprise architects could try to attribute all such failures to errors of execution rather than errors of intention or planning (in other words, the vision was okay, the overall plan was okay, but the project managers screwed up). Alternatively, they could try to find fault with each of the failed plans, identifying ways in which these plans disobeyed some of the core principles of enterprise architecture.

But I believe these examples force us to do something more fundamental - to rethink the mission of enterprise architecture. If the mission to simplify and unify conflicts with the mission to "align" (whatever that means) with the business, then perhaps enterprise architects need to find a new mission.

Simplification and unification is merely a strategy in the management of complexity. As I see it the enterprise architect's job is not to manage-away the complexity of business or IT, but to find ways to make the complexity more manageable.

More manageable for whom? Just about anybody. Business line managers need to be able to manage the complexity of their business operation, and the top management team needs to be able to manage the complexity of the whole organization. Meanwhile, business support managers (such as IT and HR) need to be able to manage the complexity of their own domain as well as providing specialist support for the rest of the organization.

Thus the enterprise architect should be helping the CIO answer the following two questions.
  • How can I manage the complexity of IT operations, systems and services?
  • How can IT contribute to managing the complexity of the rest of the organization (including other support functions)?
Similar questions apply to other support functions. Thus for example the enterprise architect should also be able to help the HR director answer the following two questions.
  • How can I manage the complexity of people management?
  • How can I help develop the capability of the whole organization to collectively manage complexity?


The IT department can support the management of complexity across the organization by providing a set of useful tools, including business intelligence tools, decision support tools, social networking tools and so on. (The provision of management information has always been a major responsibility of the IT department, and this cannot be done properly without understanding how and why management information is used.) The HR department has a different set of tools, such as training programmes, incentive packages, career paths, and so on. The enterprise architect's role here is not to manage the detailed implementation of all these different tools, but to understand the overall structural requirements and implications.


Architecture is basically about structure. Business is facing some increasing complex structural problems, and the primary task for enterprise architects is to tackle these structural problems. Tackle, not solve. The collective ability of the whole organization to manage the essential complexity of demand, from customers and the rest of the environment, depends strongly (although not solely) on the emergent structure of the organization and its systems, and this is where the enterprise architect's attention should be focused.

 

Related posts: Who architects the organization? (September 2010), Complexity is not a problem (February 2013)

Updated 16 January 2013. Links added 12 September 2021

9 comments:

  1. Richard,

    You raise important points.

    The vast majority of organisations/sectors do not know how everything is put together to make the business work.

    But in an age when most businesses can be considered to be “data refineries” how do IT practitioners simplify complexity and create business clarity? And how can stakeholders be brought onside?

    The reason so many public/private IT projects fail is that there is not enough clarity about how data flows through and between businesses - there are no standards for flows of data.

    In the past architects, engineers and scientists have spent decades creating standards and practices so that flows of water, steam, electricity, oil, components and petrol could be used safely. How a business worked was clearly understood.

    But today EAs and other practitioners cannot see clearly the interdependencies between the individual business assets that enable the flow of data.

    With that in mind, may I invite you to consider a new way of thinking called “OBASHI”?

    It shows you the “big picture” of how your business works; the assets that make it work; and the interdependencies between these assets.

    You use it to create a dynamic map of the business; to join-the-dots of people, process and technology; and see how data flows through and between them, which means you can design, optimise and monitor your business better.

    It enables communication, at all levels, in the organization because no specialist knowledge is required to understand the map of the business and where and how people, process and technology fit.

    OBASHI is not Enterprise Architecture - but it will be used by Enterprise Architects, because it simplifies complexity and it will help them answer questions like the ones you ask in your blog, and many more besides. The whole point of OBASHI is to enable businesses to make better decisions based on the best information available.

    Should you want more detail about the above may I invite you to have a look at “OBASHIThink”, a forum we launched about three weeks ago. We have just welcomed our 50th member and we hope to welcome many more. You can find it at www.obashi.co.uk

    Fergus

    ReplyDelete
  2. Thanks Fergus. OBASHI has some good ideas, and I agree that it can be useful to view the data flows through an organization, as well as the "refinery" process that transforms crude data into more useful forms. I also agree that a focus on standards and interoperability can be useful.

    However, you still seem to be pushing the simplification agenda, which my post is arguing against. You apparently want IT practitioners to "simplify complexity and create business clarity". But managing complexity does not necessarily mean managing-away complexity, it can also mean developing greater capability to manage the complexity that exists, and I think this leads us to an entirely different way of understanding the mission facing the enterprise architect. I'd be interested to know whether you think OBASHI can support this new mission.

    ReplyDelete
  3. Isn't the challenge to simplify the *communication & understanding* of complex business behaviour rather than to try to simplify the essentially complex?

    ReplyDelete
  4. Thanks Nigel. I think I see the challenge slightly differently. As Marx wrote, the philosophers have only interpreted the world in various ways; the point is to change it. Thus the enterprise architect shouldn't be communicating the business to the business (grandmothers sucking eggs anyone?) but should focus on communicating specific structural issues.

    Of course you are right that simplification is a useful communication tactic, but surely that's a means to an end not an end in itself.

    In any case, I prefer not to talk about simplification at all, but about power and intelligence. As an analogy, a theatre director shouldn't try to simplify Shakespeare, but does perhaps try to strip away anything that gets in the way of the actors powerfully expressing the rich complexity of Shakespeare.

    ReplyDelete
  5. Hi Richard/Nigel,

    Oil refineries are hugely complex. Yet they very rarely suffer failures - why?

    Because the industry understands and can easily communicate the complexity of the business.

    In Oil & Gas, digital sensors are attached to every asset, and digital flows (representing product flows) are clearly understood, valued financially and constantly monitored.

    They can see how everything is put together to enable the flow of product. Any asset that does not add value to this process is ruthlessly discarded.

    There is nothing wrong with complexity. In fact we can have as much as we want as long as it adds value and can be easily communicated to stakeholders. But obviously the more it can be simplified the better.

    Oil & Gas is complex – but it has been simplified to the maximum extent. The industry has business clarity and has arguably been the most successful market for decades.

    IT on the other hand suffers many failures because it has been unable to see and communicate how data flows through people, process and tech.

    OBASHI lets you take the tried and tested techniques of Oil & Gas and apply them to business and IT in any industry.

    So, yes, without a doubt, OBASHI can help develop a greater capability to manage the complexity that exists.

    Last week I presented at the “Data Management Knowledge Exchange” of The Petroleum Exploration Society of Great Britain.

    Rather than me droning on, perhaps the slides from the presentation would be more interesting.

    http://think.obashi.co.uk/profiles/blogs/a-conference-talk-about-obashi

    ReplyDelete
  6. Hello Richard,

    You state: "If the mission to simplify and unify conflicts with the mission to "align" (whatever that means) with the business, then perhaps enterprise architects need to find a new mission."

    In my case, simplicity and unification have NEVER been the mission of EA.

    http://bit.ly/9eyyWF

    Does that mean that I do NOT need to re-examine *my* priorities? :-)

    Simplicity and unification are good goals for a BUSINESS to have, if they want them... but not for EA.

    I believe that good EAs have a principle that we follow: that we should present the simplest, most cost effective, most feasible way for the business to achieve its desired goals.

    Simplicity moves forward when "stupid complexity" is killed off in the process... but complexity is oftn required. In some cases, it is preferred.

    Be as simple as it is reasonable to be, but no simpler.

    ReplyDelete
  7. Nick

    As you noted, what I actually said was a conditional statement: IF mission A conflicts with mission B, then perhaps enterprise architects need to find a new mission.

    However, I didn't say ONLY IF. So even if your own approach to EA is NOT based on simplification and unification, it doesn't follow from what I said that you do NOT need to re-examine *your* priorities. Maybe you do, maybe you don't.

    What I would say is that for many people, there is a critical difference between the espoused mission and the de facto mission. I'm less interested in what enterprise architects say they are doing, or should be doing, and more interested in what they actually do. Any significant difference between espoused theory and theory-in-use calls for re-examining priorities - what Argyris calls some double-loop learning.

    Nick, according to your blogpost on The Mission, Capabilities, and Business Output of Enterprise Architecture , the mission of EA involves strong influence over the structure of the business organization. So who architects the organization at Microsoft?

    ReplyDelete
  8. "But here's the problem: real business is inherently complex."

    Is it?
    Is business "inherently complex", or have businesses just allowed it to become so? Do some business managers for example deliberately strive to differentiate their activities from other business units, thus adding to the perceived complexity, whereas in thruth they are no different to others.
    In other words, just as Enterprise Architects might strive for simplicity, are there other vested interests at work who are striving for complexity?

    ReplyDelete
  9. Business CAN be complex (not always, but often enough). EAs should help the business deal with any inherent complexity, but probably couldn't and shouldn't try to remove it. Where the business has become needlessly complicated by (e.g.) obsolete processes, old ideas etc … then the goal of simplifying things is useful.
    As for EA's mission - for me it's never been about simplification and unification. It's more about helping a business fulfil its intentions; and it's WAY more than just IT :)

    ReplyDelete