Special Powers of the Architect - Abstraction

Does the Enterprise Architect have special powers?

---

One idea is that the intrinsic essence of an enterprise architect is to think about abstraction. (By the way, abstraction seems to be a category that can only be defined polythetically; it appears to have many characteristic features, but we seem to be unable to work out any defining features.)

Suppose that the use of an enterprise architect (if any) is to solve practical business problems. For this purpose, enterprise architects are interchangeable with systems thinkers, just as Jaffa Cakes are interchangeable with Fig Rolls. For a large tea party or enterprise transformation programme, we might have a plate containing Jaffa Cakes and Fig Rolls and various other cakes and biscuits.

But presumably that's not the only thing that an enterprise architect is good for.

So one theory-practice question is this. What are the essential things that an enterprise architect brings to solving practical problems? If it is abstraction, what special powers (if any) does this give to the practising enterprise architect? And what are the essential things that a systems thinker brings to solving practical problems?

Abstraction is associated with a number of myths and pitfalls

• The myth that radical structural and behavioural change can and should be driven by a few simple and powerful ideas. On the misuse of general principles (Jan 2012)

• Valuing abstraction and adaptability above everything else. Avoiding thinking about mundane technology when building pure business models. IT Innovation at Small Bakery (April 2009)

• The relationship between the general and the specific is poorly supported by the prevailing tools and methods, which often reduce the question to simplistic abstraction hierarchies. TOGAF 9 - Enterprise Continuum (Feb 2009).  Instead of generalizing everything as much as possible, architects need to reason explicitly about system structure and dynamics - cohesion and coupling, composition and decomposition, change and emergence. They need to understand granularity and stratification as active choices, rather than text-book patterns. Architecture and Abstraction (May 2004)

• Mainstream EA frameworks typically appear as premature codifications of ungrounded abstractions. The attempted diffusion of these codified abstractions is then ineffectual, because these ungrounded categories lack any relevance to real business challenges. ... So instead of discussing concrete business problems, we find ourselves discussing generic categories of business problem. Towards Next Practice EA (May 2013) 

See also

• Special Powers of the Architect - Getting The Big Picture (Dec 2010)
• Special Powers of the Architect - Judgment (January 2012)

Updated 26 October 2016

On the misuse of general principles

#entarch There is a common fallacy among enterprise architects that radical structural and behavioural change can and should be driven by a few simple and powerful ideas. Alas, the public sector is strewn with the disastrous consequences of this fallacy.

We can find countless examples from the National Health Service (NHS) in the UK. For Steve Harrison, Honorary Professor of Social Policy at the University of Manchester, the idea that NHS reorganisations can be triggered by a few general ideas is one of the Seven Fallacies of English Health Policy. He points out that high levels of abstraction (beloved by academics and architects alike) do not allow proper assessment of the plausibility of claims about benefits of reorganisation and how the system will work. (HT @mellojonny)

Where do health reorganization principles come from? I asked a popular search engine, and was led to a paper called Basic Principles of Information Technology Organization in Health Care Institutions (JAMIA 1997); (I suppose from the high search ranking of this paper that it is a widely used source for such principles.) The paper concludes that all organizations MUST have certain characteristics, based on a single case study where these characteristics seemed to be beneficial; in other words, arguing from the particular to the general. (I'm sure there must be some more rigorous studies, but they don't seem to get as good search rankings for some reason.)

But many of the principles that govern sweeping architectural reforms of the public sector aren't even derived by thinly based generalization from such observed vignettes, but are derived from purely abstract concepts such as "choice" and "competition" and "justice", to which each may attach his or her own politically motivated interpretation.

This leads to several levels of failure - not only failure of execution and planning (because the generalized principles are not sufficiently refined to provide realistic and coherent solutions to complex practical problems) but also failure of intention (because a vague but upbeat set of principles helps to conceal the fact that the underlying vision remains woolly).

Conceptual Modelling - Why Theory

@dougnewdick asks Why Bother With Concepts? In his answer, he quotes the Gang of Four song Why Theory. Each day seems like a natural fact/but what we think changes how we act.

I think there is a critical difference between two interpretations of the term Conceptual Architecture. I'd describe Doug's sample conceptual architecture as essentially a component architecture, which happens to be at the conceptual level of abstraction. In other words, it provides a conceptual understanding of how the components work together.

That's not at all the same as understanding how the concepts themselves work together, which is what I think one needs to satisfy the Gang of Four requirement for an architecture-of-concepts, and which would also be implied by Doug's reference to the Wikipedia article on Philosophical Analysis.

The Wikipedia article mentions critiques of the traditional approach to philosophical analysis by Quine and Wittgenstein. I have used some of their ideas - although with no claim to profundity - in my own approach to conceptual modelling. See for example my 1992 book on Information Modelling.

To take a concrete example, if we are going to build systems and management practices to anticipate competitor behaviour, it might help have to have a robust concept of COMPETITOR. I once worked with a company that thought it knew which its competitors were, and were quite indignant when a competitive threat appeared from an entirely different quarter. Traditional philosophical analysis implies you can fully characterize a concept like COMPETITOR in advance, but Wittgenstein and Quine teach us to be cautious of monothetic, appearently objective conceptual definitions.


One of the problems with conceptual models is that they are often too generic, and therefore insufficiently meaningful. Anyone who has ever been on a data modelling course can produce a model with CUSTOMER and ORDER and PRODUCT. But proper conceptual analysis entails taking concepts apart to understand how they work in a particular business context. Where do products come from, what does it mean to be a product, how do products evolve, what are the essential and non-essential differences between products? Why do we have this many products, and what are the forces that would result in increasing or decreasing the number of products? What does the business know about products, and what does the business want to know?

Doug's post was prompted by Peter Bakker's post Factual Architecture, which expressed a concern that concepts can be too abstract and unverifiable – failing to connect with audiences, failing to connect with reality. The way that some architects try to verify concepts is to demonstrate how the concepts could be represented in some technical system - but I prefer to encourage stakeholders to verify concepts in the business domain itself. This leads me to a slightly different notion of Factual Architecture, which would provide an empirical business context in which the business concepts and knowledge can be grounded.

Is this really the way it is? Or a contract in our mutual interest? (Gang of Four: Contract)

See also Big Data and Organizational Intelligence (November 2018)

Model-Driven Architecture?

@greblhad suggests building Lego models as a technique for enterprise architecture: "If your #entarch can not be illustrated in 3D using LEGO then you have a problem".

The reason Lego cannot reliably express architecture is because just looking at the model doesn't tell us which elements are architecturally significant. For example, if a particular Lego model has a row of small bricks over the window instead of larger ones, and uses green bricks for the base, how can we know whether the architect actually specified this detail, or whether the architect left these details unspecified and the person building the model simply used the bricks that were available? In other words, the same physical model could have been produced by two different architectures: one including the instruction "Use small bricks over the window and green bricks for the base", and one including the instruction "Don't buy more bricks if you don't have to". (The latter being derived from one of the popular architectural principles that supposedly drive architectural practice.) Note that in this example, the two possible instructions are at two different logical levels, and the illustration may mislead us about the architect's real intentions. For a different example, see my post What if architects designed our communities?

At its best, LEGO (even with people figures, as suggested by @tetradian) shows a momentary instantiation of an architecture, and is therefore at the wrong level of abstraction.

I accept that we are not talking about using Lego models as a specification tool. But illustration only works if you know what is being illustrated, as Wittgenstein pointed out (Philosophical Investigations). Furthermore, the representational nature of the model may be problematic. If Lego only has a limited palette of colours, do we interpret green to mean any shade of green, or that particular shade? Does the use of green bricks in the model indicate the use of green bricks in the planned structure, or does it have some other meaning, for example indicating the need for specially treated bricks? And so on.

@greblhad was inspired by LEGO's new "product", called LEGO® Serious Play®. Perhaps enterprise architecture needs creative play as well as rigorous specification, but it would be surprising if the same tools and methods supported both.

IT Innovation at Small Bakery

Can your IT department innovate like this bakery? asks Mark Raskino (Gartner) via Sally Bean. Mark describes an East London bakery that uses Twitter to notify its customers whenever freshly baked bread comes out of the oven.

The system involves a specially designed wall-mounted device that the bakers can operate with one hand, while holding a tray of bread with the other.

Mark clearly doesn't think most IT departments would be comfortable with this kind of system. So I asked Mark and Sally who would lead that kind of innovation in a typical IT department. Would it be the business analysts? The enterprise architects?

The trouble is that many people in those kind of IT roles have been trained to value abstraction and adaptability above everything else, and to avoid thinking about mundane technology when building pure business models. But how often does that kind of abstract thinking produce concrete innovations like this bakery? In many organizations it is quite the opposite - technology-driven opportunistic change to the business model is discouraged because it runs contrary to prevailing thinking. And as Mark points out, corporate IT departments don't like purpose-designed hardware. This system is not designed for adaptability, it is designed to improve the business.

So this project was driven by a marketing agency. Business innovation may depend on the latest technology; but wouldn't it be a disgrace if it turned out that the most innovative companies are those too small to have an IT department?

TOGAF 9 - Enterprise Continuum

The concept of Enterprise Continuum was present in TOGAF 8, but it is a little clearer in TOGAF 9.

What is an Enterprise Continuum? In TOGAF 8, it was defined as "a virtual repository", but this wasn't very clear. In TOGAF 9, it is now defined more explicitly as "a view of the Architecture Repository that provides methods for classifying architecture and solution artifacts as they evolve from generic Foundation Architectures to Organization-Specific Architectures". In other words, it is not the repository itself, but a way of classifying the content of the repository.

The relationship between the generic and the specific has always been a critical dimension of architecture, as I've argued on this blog many times (see label: generic v specific). Enterprise Continuum provides a framework for architects to reason about the right level of generality or specificity for a given artefact.

Here are some implications of this framework.

• Progressive Refinement - Many methods assume progression from the generic to the specific, sometimes known as "refinement". But this is not the only possible approach, and is not always the most appropriate. (See my posts on Progressive Design Constraints and Analyzing the Rusty Lawnmower.)
  

• Management of Variation - Capabilities can often be decomposed into highly generic elements and asset-specific elements. (See my example of Green Coffee Beans.) Decoupling these capabilities helps to drive higher levels of cross-process and cross-organizational sharing. With SOA, this can be supported by separate architectural layers. (See my posts on Homogeneous Business Vocabulary and Merging Capability Modelling with Process Modelling.)

• Adaptation and Adaptability - To achieve agility or flexibility, an enterprise or system or solution may need to be under-determined. This is sometimes confused with abstraction. (See my posts on Adaptation and Adaptability.)
  

• Strategic Differentiation - Enterprise architects need to understand the ways in which a given company is similar to any other company in the same industy sector, and what particular things differentiate this company from other companies. (See my posts The General and The Particular and Service Planning.)

But although this is a critical question for architects, it is poorly supported by the prevailing tools and methods, which often reduce the question to simplistic abstraction hierarchies. Let us hope that TOGAF 9 stimulates further development in this area.

Hypothetical Business Models

Robin Meehan (Smart421) asks a question about modelling time sequence dependencies in EA modelling tools

"I often want to model ‘what if’ scenarios, i.e. what the various architecture domains would look like if we proceeded with Option A, or B etc ... [but] I don’t want my various different options to ‘bleed’ into each other in the model. ... The only way I’ve found round this in the past is to duplicate parts of my model to keep them separate (in different packages etc) - which is a bit ugly."

Of course, the problem Robin identifies is more than just time sequence dependencies. Sometimes we have to explore the differences between A-followed-by-B and B-followed-by-A, but there may be many other permutations to explore. If you have to duplicate parts of the model for each permutation, this severely limits the number of permutations you can compare.

Many architects will try to solve this kind of problem by abstraction - coming up with a model that abstracts away from the differences between the options, so that the model supports all the possible options and all the possible permutations between the options. However, the standard model isn't then much good for comparing the options. In some cases it may be worth building a flexible capability that can support both A and B, but how do I tell whether this is one of those cases?

(Michael Bell's SOMF modelling approach does allow AS-IS and TO-BE to be shown on a single diagram, but I don't know how practical it is to put multiple alternatives on a single diagram.)

What Robin highlights here is that the standard modelling tools and notations aren't always very helpful at supporting choice, especially when the choices get at all complex. Often it seems that the tools only really work for documenting the architectural and design choices that have already been made, rather than for making the choices in the first place.

The best answer I can give Robin is to use the available tools, but to adopt a different modelling style. But I need a good case study to work on. Contributions welcome ...

Architecture and Abstraction

SOA calls upon designers and architects to operate at a higher level of abstraction.

One mode of abstraction commonly associated with information architects is generalization - they are often thought to operate mainly with generalized business objects/processes, such as CUSTOMER and SALES.

But this is actually not where the real architectural challenges lie.

Instead, architects need to reason explicitly about system structure and dynamics - cohesion and coupling, composition and decomposition, change and emergence. They need to understand granularity and stratification as active choices, rather than text-book patterns.

But the generally available architectural practices and modelling notations simply do not support these challenges. How are architects supposed to pay attention to such concerns as coupling and flexibility, when they are working with models that don’t make these concerns visible, and with practices that don’t support reasoning about these concerns?

We are developing tools and techniques to support this reasoning, and the development of architectural knowledge and know-how.