Emergent Architecture

#entarch #emergence #systemsthinking What is emergent architecture, and what are the practical implications for enterprise architecture?

In August 2009, Gartner produced a definition of Emergent Architecture with two synonyms (Middle-Out EA and Light EA), two characteristic practices (modelling lines not boxes, modelling relationships as interactions) and seven characteristic properties (non-deterministic and decentralized; autonomous, rule-bound, goal-oriented and locally influenced actors; dynamic or adaptive systems; resource constraints) [Gartner Identifies New Approach for Enterprise Architecture, August 2009].

At the same time, Dion Hinchcliffe produced a similar list of properties and one of his trademark diagrams [Pragmatic new models for enterprise architecture take shape, August 2009]. Meanwhile Dion was also talking a lot about WOA (which he credits to Nick Gall of Gartner), and there was clearly a link in their thinking between WOA and some notion of emergence [A Web-Oriented Architecture (WOA) Un-Manifesto, December 2009].

Overall, the emergent and evolving definition of Emergent Architecture across the internet is pretty muddled: although other writers in the Enterprise Architecture world may reference Gartner and/or Hinchcliffe, they don't always pick up the full richness and power of their definitions. In addition, these writers may be influenced by the agile community, where emergent is contrasted with upfront and seems to mean something like making it up as you go along.

For example

The architect should collaborate with the development team to define and code higher-level contexts, responsibilities, interfaces, and interactions, as needed, and leave the details to the team. The development team, through the rigorous use of automated unit and story tests via continuous integration, is then able to improve the system design incrementally and continually—both within and across model-context boundaries— without compromising system functionality. Gartner uses the term Emergent Architecture to describe this practice. Keeping Architectures Relevant, Microsoft Architecture Journal

The practice described in that article can only be described as emergent on a fairly narrow interpretation of the word, presumably based on the rule-bound criterion and ignoring the other characteristics.

My own view is that all of the characteristics Gartner and Dion Hinchcliffe originally proposed (with the possible exception of resource constraints, which are nothing new) could be regarded as emerging, in the sense of being new and trendy and representing a departure from the modernist paradigm of traditional enterprise architecture. (See my post on Modernism and Enterprise Architecture.) But not all of the characteristics they proposed are directly associated with emergence, in the complex systems sense.

Wikipedia defines emergence as the way complex systems and patterns arise out of a multiplicity of relatively simple interactions [Wikipedia: Emergence]. As I see it, the notion of emergence leads to a key distinction for enterprise architecture, between a planned order (which Hayek called Taxis) and an emergent spontaneous order based on self-organization (which Hayek called Cosmos).

Although some writers like to use biological and ecological analogies when talking about emergence, the sociopolitical analogies are probably more relevant to enterprise architecture because they include the notion of human intentionality. (And some people use the terms top-down and bottom-up, but nowadays I try to avoid these terms because of their potential ambiguity. See my note on Service Planning.)

The distinction between planned and emergent architecture is akin to the distinction introduced by Henry Mintzberg between deliberate and emergent strategy, the latter referring to strategies that originate in the interaction of an organization with its environment [Wikipedia: Strategy dynamics].

This distinction also maps onto some recent work in the system-of-systems (SoS) domain. Mark Maier introduced a distinction between directed and collaborative systems (1998), and this has been developed by the U.S. Department of Defense (DoD) into a more complicated four-part schema (directed, acknowledged, collaborative and virtual). See for example System Engineering Guide for System-of-Systems Engineering (Version 1, August 2008). Planned architectures tend to assume directed or acknowledged systems-of-systems, while emergent architectures are associated with collaborative and virtual systems-of-systems.

Boxer and Garcia write:

In the complex systems-of-systems contexts supporting distributed collaboration, the architecture of the collaborative enterprise has to be approached as emergent, created through an alignment of individual architectures. [Enterprise Architecture for Complex System-of-Systems Contexts, 3rd Annual IEEE Systems Conference in Vancouver March 23-26 2009 (abstract) (pdf)] See also Type III Agility and Ideologies of Architecture.

The acknowledged category of systems of systems allows for some flexibility and autonomy for local development of component systems, while remaining under central oversight and direction. This is surely where rule-bound actors would fit, and would include the practices described in the Microsoft Architecture Journal article mentioned above (Keeping Architectures Relevant). There is also some scope in the acknowledged category for dynamic or adaptive systems and adaptive processes. But this is some way short of a genuinely emergent architecture.

Gartner is closer to the mark when it talks about decentralized decision-making (which it calls non-deterministic) involving autonomous goal-oriented actors, responsive to local influence. Similarly, Dion Hinchcliffe talks about community-driven architecture run by autonomous stakeholders, producing decentralized solutions with emergent outcomes.

I don't know whether Gartner is still promoting the idea of emergent architecture, or whether its definition has evolved since 2009, given that Nick Gall's recent work (such as his latest piece on Panarchy) doesn't seem to use the term at all. However, Gartner's original piece on emergent architecture remains available on its website, and continues to be referenced as one of the primary sources for the term.

---

What are the practical implications of emergence for enterprise architecture?

One of the most important practical differences between planned architecture and emergent architecture is that we tend to think of planned architecture as a design-time artefact - something that is envisioned, designed and then implemented by architects. So architecting is regarded as a special kind of designing. Implementation may be directly managed by the architects, or indirectly by means of a set of architectural policies to govern acquisition, development and use.

The implementation of a planned architecture always involves a gap between plan and reality. Plans take time to realise, some bits of the plan may never get realised. There is always a defacto architecture, which we may call architecture-in-use, which is a structural description of what-is-going-on, paying attention to certain structural and sociotechnical aspects of a (run-time) system of systems from a given observer position.

Devotees of planned architecture see this in terms of a simple transformation from AS-IS to TO-BE. Their attention to the existing defacto architecture is largely motivated by the need to define a business case and a technical transition strategy for replacing AS-IS with TO-BE, possibly in discrete phases or chunks.

But emergence tells us this: that the architecture-in-use emerges from a complex set of interactions between the efforts and intentions of many people. The architects cannot anticipate, let alone control, all of these interactions. There may be some areas in which the architects are able to carry out something that looks a bit like design, either autonomously or collaboratively, but there will be other areas in which the architects are simply trying to understand the structural implications of some messy situation and make some useful interventions. The primary activity of architects is therefore following not a design loop but an intelligence loop.

However that depends what we mean by design. There is a renewed interest in a generalized notion of design in the enterprise architecture world, especially with the current popularity of Design Thinking (and such derivatives as Hybrid Thinking).  But that's a subject for another post.

---

Sources

Gartner identifies new approach for enterprise architecture (Aug 2009)

Commentary on Gartner's emergent architecture concept by Abel Avram, Leo de Sousa, Adrian Grigoriu and Mike Rollings (then with Burton Group).

Dion Hinchcliffe, Pragmatic new models for enterprise architecture (Aug 2009)

Mark Maier, Architecting Principles for Systems of Systems (InfoEd June 1997)

See also

Peter Cripps, Enterprise Architecture is Dead. Long Live Interprise Architecture (Oct 2010)

Tom Graves, Hybrid-thinking, enterprise architecture and the US Army (May 2010) 

Three or Four Schools of Enterprise Architecture (December 2011)