Towards Next Practice EA

A few weeks ago, @Cybersal and I met with @snowded to talk about enterprise architecture. He showed us a graph of the Complex Space, whose two dimensions were Evidence and Consensus. Dave has since posted a version of this graph on his blog.

Source: Dave Snowden, Cognitive Edge April 2013
Used with permission

At present, I believe there are two main clusters of Enterprise Architecture practice:

Mainstream EA. This is where TOGAF and Zachman and all the other popular frameworks sit. These frameworks are largely based on a 1980s view of enterprise and IT, contain apparently inflexible classification schemes, and are strongly aligned to the commercial interests of the large software companies and system integrators (IBM, Cap-Gemini, and the rest). These frameworks sit in the top left corner of Snowden's chart, with a high level of consensus and a very weak evidence-base.

If we look at Mainstream EA in terms of Boisot's I-Space, I would suggest these frameworks appear as premature codifications of ungrounded abstractions. The attempted diffusion of these codified abstractions is then ineffectual, because these ungrounded categories lack any relevance to real business challenges. From time to time, practitioners may well be able to do some useful work, and they may even be persuaded that their success is a consequence of using these frameworks. But it is also possible that these occasional successes are despite the framework and not because of it.

Next Practice EA, with a smaller but growing number of practitioners and researchers proposing radical alternatives to Mainstream EA. This community includes some well-placed industry analysts (such as Nick Gall of Gartner) as well as a significant number of independent consultants. Many of these alternatives involve hybrids of EA with some other discipline, such as Systems Thinking or Design Thinking. However, the evidence base for these hybrids is if anything even weaker than for Mainstream EA frameworks, and there is no consensus whatsoever. Few if any of these hybrids have been tested in full, and sometimes they appear to consist of little more than a bricolage of ill-assorted concepts and techniques, which I call Methodological Syncretism.

What would it take for Next Practice EA to develop either sufficient evidence or sufficient consensus to represent a serious challenge to Mainstream EA? It seems to me that the answer lies at the Concrete, Undiffused, Uncodified corner of the I-Space cube. What I have found highly frustrating in many discussions with both enterprise architects and systems thinkers is an apparent belief in the absolute virtues of abstraction. So instead of discussing concrete business problems, we find ourselves discussing generic categories of business problem. Many of the discussants either don't appreciate the difference between "Cost Saving" and "Cost Saving at Smartchester College", or they think that discussing the general category is more intellectually respectable than discussing a specific case.

There is of course a reason why discussing specific cases is treated with disdain in some quarters, which has to do with the way cases are used in some business schools, notably Harvard Business School. There are two ways to explore a specific case. In the Harvard approach, as I understand it, students are presented with cases which they need to "solve", using a light sprinkling of theoretical concepts. You use the theory if it works; if it doesn't, you ignore it. This approach isn't designed to produce a deeper understanding either of the cases or the underlying theory. The alternative (which I prefer) is a much more reflective and reflexive approach, where you use the case as an instrument to understand and test and refine the underlying theory. The aim is to end up with a rich and specific narrative that is both empirically and theoretically grounded. Only when we have a sufficient number of these narratives do we begin, with great caution, to generalize from them. But that's a lot harder and more time-consuming than putting together a blog or presentation full of post-modern cleverness. (Okay, I can be as guilty of this as anyone else.)

---

This is an extract from my draft eBook Towards Next Practice Enterprise Architecture, available from LeanPub.

These and related issues were discussed at a public meeting Perspectives on Enterprise Architecture and Systems Thinking, in Central London, June 14th 2013. For my own presentation slides, subsequent notes and links to reports by other participants, see Schism and Doubt (July 2013).

Differentiation and Integration

In my post on Business Design Choices, I suggested that the real challenge for business architecture was to appreciate the economic and social impact of structure. I identified some advanced business design choices where business architecture has a useful role to play, and where some IT people might have some relevant aptitude. In this post, I am going to expand on one of these.

---

Where does standardization make sense, and where should requisite variety be deployed? This question goes back to the work of Lawrence and Lorsch (L2), and has recently been rediscovered (in slightly different terms) by Ross, Weill and Robertson (RWR).

In their book Enterprise Architecture as Strategy, RWR define something they call an Operating Model, with two independent dimensions, business process standardization and integration. "Although we often think of standardization and integration as two sides of the same coin, they impose different demands. Executives need to recognize standardization and integration as two separate decisions." (p27)

Many people in the IT world take for granted that standardization (reduction in variety) is a good thing.  RWR acknowledge the benefits of standardization not only in terms of throughput and efficiency, but also predictability. However, they point out the potential downside of standardization both as a state (standardized processes limit local innovation) and as a state-change (politically difficult and expensive to rip out and replace perfectly good and occasionally superior systems and processes).

Integration, which RWR define primarily in terms of data sharing, is also assumed to be a good thing. RWR identify the benefits of integration in terms of efficiency, coordination, transparency and agility, and acknowledge the challenge of integration as a state-change in terms of "difficult, time-consuming decisions".

The two dimensions of standardization and integration produce a two-by-two matrix as follows.

Operating Model Quadrants (Adapted by Clive Finkelstein from Figure 2.3 of “Enterprise Architecture as Strategy”) - Source The Enterprise Newsletter #38.


Although RWR are careful to present a contingency theory of enterprise strategy, in which any of these four operating models may be strategically valid, the conventional rhetoric of the two-by-two matrix places the preferred strategy into the top right quadrant - thus Unification. Many enterprise architects from a traditional IT background may feel most comfortable with the Unification quadrant. (There may be a process of idealization here - see Schwartz.) Indeed, RWR go on to present an Enterprise Architecture Maturity Model, which starts with the easy bits of enterprise architecture (where things are already Unified) and ends with the challenging bits (where things are or should be Diversified).

---

L2 also identify two dimensions, which they call differentiation and integration. They tend to see increasing differentiation as a healthy response to increasing opportunity and complexity - a growing organization in a growing market - "faster change and greater heterogeneity" (p 235). Differentiation is not merely a difference in working practices, but includes at its core "the difference in cognitive and emotional orientation among managers in different functional departments" (p11).

Integration is then the administrative response to this increasing differentiation - how to maintain the overall coherence and viability of the enterprise as a whole. Integration is defined as "the quality of the state of collaboration that exists among departments that are required to achieve unity of effort by the demands of the environment" (p11). The topic of integration covers both the organizational state and the mechanisms to produce and support this state. For L2, the challenges of integration are not primarily IT related (data sharing) but personal and political (conflict resolution).

L2 don't offer a two-by-two matrix of Differentiation and Integration in their 1967 book (I guess the two-by-two matrix hadn't then established itself as an essential consultancy tool), but if they did then presumably the top-right quadrant would be High Differentiation, High Integration. This very roughly corresponds to what RWR call Coordination.

But in any case, a two-by-two matrix would be misleading. The point isn't to choose whether you have differentiation and integration or not; the point is to determine how much and what kinds of differentiation  and integration you need. L2 are explicit in their support of contingency theory (different strategies being appropriate for different organizations depending on environmental factors). The more complex and dynamic the environment, the greater the need for differentiation and integration.

If we are to take business architecture seriously as a discipline, then this kind of question is clearly central. In his post on Contingency Theory and Enterprise Architecture, Andy Blumenthal argues that contingency theory entails keeping your options open, but sometimes it just means making appropriate strategic choices. If the slogan "enterprise architecture as strategy" is to mean anything at all, then surely this is what it means.

---

Lawrence, P., and Lorsch, J., "Differentiation and Integration in Complex Organizations" Administrative Science Quarterly 12, (1967), 1-30. (see summary here)

Paul R. Lawrence and Jay W. Lorsch, Organization and Environment, Managing Differentiation and Integration. Harvard University 1967.

Jeanne W. Ross, Peter Weill and David C. Robertson, Enterprise Architecture as Strategy. Harvard Business School Press 2006.

Howard S. Schwartz, Narcissistic Process and Corporate Decay – The Theory of the Organizational Ideal. 1990. See his paper On the Psychodynamics of Organizational Disaster (Columbia Journal of World Business, Spring 1987) See also Joe Bormel's blogpost on Narcissism, Oxygen and HCIT Vision (June 2009).


Related posts

Business Design Choices (January 2010)
EA Effectiveness and Process Standardization (August 2012)

Ecosystem SOA

The SOA world is finally catching up with some of the ecosystem ideas that I published in my 2001 book on the Component-Based Business (see my Slideshare presentation) and developed further in several articles and presentations for the CBDI Forum over a number of years.

• Supply and Fit (September 2000)
  
• Identifying Components 2 (October 2001)
• Opportunities in the Insurance Ecosystem (December 2004)
• Business Strategy Planning for the Service Economy (May 2006)
  

The biological approach to creating business and software services is radically different to the solution-driven approach, and is based on biological and ecological metaphors.

• First we identify an ecosystem, which may contain both human users and existing artefacts.
• Then we identify services that would be meaningful and viable in this ecosystem.
• Then we procure devices that enable the release and delivery of these services into the ecosystem.

I previously defined Three Types of Requirements Engineering, and we can map these onto different styles of SOA.

Solution-Driven (Specific)	Solution-Driven (General)	Evolution-Driven
Identify Business Problem Identify "Users" Negotiate Requirements Define Solution	Identify Domain Identify Domain Experts Define Requirements Design Solution Kit	Identify Ecosystem Identify Services Procure and Release Devices
Experimental SOA	Enterprise SOA	Ecosystem SOA

(Some people use the term Web Oriented Architecture (WOA) for what I'm calling Ecosystem SOA.)


If we regard these as phases of maturity, then we can have a straightforward roadmap from left to right, as in for example the CBDI SOA Roadmap. However, some organizations may need to tackle these styles of SOA in parallel rather than in sequence.

---

A service portfolio plan for Enterprise SOA can be based on an enterprise model that identifies the capabilities of the enterprise and clusters these into domains. In the CBDI Forum's SAE methodology, the domains are classified as Core and Contextual according to a matrix derived from Geoffrey Moore. (Note how the domains migrate around the matrix over time.) See for example my blogpost Tesco outsources core eCommerce.



A similar model, derived from Amin and Cohendet's book Architecture of Knowledge, explicitly describes Core and Periphery in terms of knowledge intensity. In other words, the reason something is classified as Core is because it encapsulates some important (strategic) knowledge.

For example, an insurance company knows more about insurance than about cars, so when providing car insurance it may decide to partner with other organizations that know more about cars than about insurance. This results in a composition of insurance-related services and car-related services (for example, determining the insurable value of a used car). Such compositions can be either directed (in other words, composed by a single dominant player) or collaborative (in other words, emerging from the interaction between multiple players within the ecosystem).

For example, an online retailer knows about her products and services, but doesn't wish to become an expert in credit card handling, or to be responsible for data protection and security, so she delegates these concerns to a specialist provider that knows more about these matters.

Similar considerations can apply to industry consortia, such as ACORD (for the insurance market). ACORD can define generic service-based assets (such as models, schemas, interfaces and so on) for insurance. However, insurance companies will also wish to use generic service-based assets to cover requirements that are not insurance-specific, such as customer management or complaints handling, where ACORD may not be able to add any knowledge-value, and it would be appropriate for ACORD to regard these as peripheral to its own activities rather than core.

---

So one approach to Ecosystem SOA is to push out from the enterprise into the ecosystem. John Hagel calls this Inside-Out Architecture, which he contrasts with Outside-In Architecture. (See my post on Outside-In Architecture.)


An Outside-In Architecture starts with a model of (the flows of) knowledge and value in the ecosystem as a whole. The strategic question for an enterprise is how to find way of both contributing value to the ecosystem, and drawing value from the ecosystem, through the provision of ecologically viable services.



For example, a telecoms company might reasonably consider that its core competence is something to do with communications. So a positional strategy would drive it to a dominant position in the middle of a large communication ecosystem, providing a platform of services that add value to a diverse range of communication activities by other people. The dominant position would allow it to negotiate a strong share of the value generated.

However, respect for the ecosystem would lead it to leave sufficient value to third parties to maintain the economic health of the remainder of the ecosystem. Instead of a simple positional strategy, a relational strategy (based on mutual trust with ecosystem partners) should produce a more sustainable and ecologically sound ecosystem.

---

Service Ecosystem from Richard Veryard


Related post: Ecosystem SOA 2 (June 2010)

Tesco outsources core eCommerce

In March 2009, Nick Lansley explained the background to Tesco.com adopting ATG e-commerce platform.

Here are some of the key points.

1. For years Tesco differentiated itself from the competition by writing its own e-commerce software that fitted in perfectly with its processes.

2. Competitors who tried to copy this software were unable to replicate its internal functionality. (Nick refers to "special algorithms".)

3. But the "core systems" are not Tesco's "core business". So Tesco has decided to outsource.

4. Tesco selected an e-commerce software platform that matches its core systems the closest.

5. With the core already there, Tesco will devote its IT staff to designing and developing new business improvements and ideas.

---

Analysis

This story fits with the view of "services on the fault line", which the CBDI Forum adopted and adapted from Geoffrey Moore. Capabilities migrate around the grid. The core e-commerce platform is migrating from top-left to bottom-right; meanwhile Tesco is devoting its internal IT resources to generating new functionality from bottom-left.



The classification of capabilities depends partly on the link to business outcomes / value. It also depends on the knowledge associated with each capability. Correct decoupling between capabilities depends on encapsulating the knowledge (know-how) embedded in each capability. (This is a version of the well-known architectural principle: separation of concerns.)

So we can understand the strategic cycle of capabilities (core and contextual) proposed by Geoffrey Moore, by reference to a model of the knowledge cycle proposed by Max Boisot.



I don't know whether the "special algorithms" remain proprietary to Tesco, or whether they (or at least the "commodity" provided by these algorithms) become available to other users of the same platform. In any case, I guess Tesco will be looking to develop new and more sophisticated algorithms.

---

See also

CBDI Forum, SOA Fundamentals (2009) - key extract from page 222

Richard Veryard and Philip Boxer, Metropolis and SOA Governance (Microsoft Architecture Journal 5, July 2005). Philip Boxer and Richard Veryard, Taking Governance to the Edge (Microsoft Architecture Journal 6, August 2006)

Richard Veryard, Business Strategy Planning for the Service Economy (CBDI Journal May 2006)

In his post Managing over the whole Governance Cycle (April 2006), Philip Boxer extends the Boisot model to produce a governance cycle appropriate for the platform-based business. See also my post on Knowledge and Culture (April 2006).

Related posts: Service Planning (December 2006), Ecosystem SOA (October 2009) Ecosystem SOA 2 (June 2010)

Updated 13 October 2015. Links updated 18 November 2019

Service-Oriented Business Intelligence

What can SOA offer to business intelligence?

Five Types of BI

The BI vendor MicroStrategy has identified five types of business intelligence, which are (not surprisingly) covered by its own BI tools. The five types are differentiated in two ways: firstly by the number and range of users, and secondly by the degree of analytical sophistication and user interactivity. According to MicroStrategy, these two are inversely proportional – thus the greater the number and range of users, the lower the sophistication and interactivity. MicroStrategy's five types of BI can therefore be arranged in a 2x2 matrix with the top right quadrant empty.

This reflects a common attitude about BI – that because the complex stuff is hard to use, and costly to service, therefore it is only going to be used by a small number of highly skilled users working on stand-alone problems.

The complex stuff is also hard to specify. Traditional system development methods are not very helpful in defining the more complex information needs, but we are looking at alternative methods of requirements analysis. See my previous post on Business Intelligence Requirements (Sept 2005).

Closed-Loop, Holistic and Real-Time BI

In June 2003, I produced a report for the CBDI Forum on Web Services for BI, which talked about closed-loop business intelligence. Among other things, this includes the ability to generate derived data objects (such as customer clusters), which can be made available for further BI enquiry or even automatically re-input into the operational information systems. One way of doing this is to wrap BI enquiries as web services, which can then be invoked from the operational applications. This is known as Embedded BI.

Anant Jhingran, IBM's director of business intelligence, made the following points to journalists last summer.

• it's not enough for an ETL tool to just grab data and dump it in a warehouse; you need the data that has made it into the warehouse, and the data that might be part of the business processes
• you also need data that is outside traditional sources and outside the enterprise
• current information is important; putting it into historical perspective is equally crucial
• business process monitoring and business activity monitoring is about a holistic perspective on the historical and current data. It's all about synergies between the warehouse and business processes. It's all about unification around XML because XML is the way business processes are communicating the data

As reported by Rich Seeley (BAM goes XML. ADTmag, June 2004), Jhingran doesn't regard BI as real-time unless it is also holistic. Although in theoretical terms you can have one without the other, in practical terms it probably doesn't make sense to separate them. So let's call this Integrated BI. Service-oriented technologies (including web services, XML and grid) undoubtedly have a useful role in implementing Integrated BI.

SOBI Roadmap

We can identify four general ways of managing and implementing BI, as follows.


1. Stand-Alone BI

BI activities are decoupled and desynchronized from the business and from operational systems. BI activities are performed by independent knowledge workers.

Current BI practice?

 
2. Embedded BI

• BI enquiries are wrapped as services and invoked from the operational systems. 
• For example, calculating the appropriate credit limit or policy for a specific customer in real time, based on a 360 degree picture of the customer and his recent behaviour.

Short-term SOA opportunity?

3. Integrated BI
 

• BI activities are coordinated with one another, and synchronized with business and operations.
• Faster and more effective business response to external change
• Greater consistency and coordination of planning effort across the enterprise.

Medium-term SOA opportunity?

4. Collaborative BI

• Orchestrate BI services across a federated management or governance structure. 
• Collaboration between knowledge workers.

Longer-term SOA opportunity?

I'd like to develop this roadmap further, with the help of my readers. I should be most interested to hear from vendors and others. Who in your organization is doing this stuff? Please let me know about your products, projects and future plans.

---

Service Oriented Business Intelligence from Richard Veryard

Articles:
Web Services to Improve Business Intelligence (CBDI Journal, June 2003)
Service-Oriented Business Intelligence (CBDI Journal, October 2005)

Related posts:
From Business Intelligence to Organizational Intelligence (May 2009)
From Networked BI to Collaborative BI (April 2016)