Is Enterprise Architecture a Science?

asks @RSessions (Roger Sessions)

I have long argued that the answer is no. What passes for knowledge in enterprise architecture is largely a combination of anecdote and received opinion. Intellectual effort is devoted to hierarchical forms and elaborate classification schemas, based on abstract reason rather than empirical measurement; this kind of work looks more like mediaeval scholastic philosophy than modern science.

In comparison, the followers of Christopher Alexander seem like nineteenth century gentleman scientists, carefully collecting specimens from which they try to infer useful principles and patterns. Alexander's four-volume masterpiece on the Nature of Order is a brilliant and fascinating work, which I think every enterprise architect should read (in their spare time), but it's not exactly suitable as an everyday handbook.

In order for enterprise architecture to qualify as a science, it has to follow scientific method. Now I am pretty broadminded about what counts as scientific method, but it's more than mere predictability. (Card games are predictable, but that doesn't make cribbage a science. Roger says if card games were predictable, he'd be a rich man. But if card games were not predictable, the casinos would go broke.)

Anyway, you can get predictability from art. I saw Dave Crosby on a documentary once, criticizing the fact that the Eagles' concerts were note-for-note predictable, CSNY preferring a slightly looser style in response to each audience. So what if EA's an art, asks @HotFusionMan (Al Chou).

Peirce understood the limitation of scientific method, holding "that slow and stumbling ratiocination can be dangerously inferior to instinct, sentiment, and tradition in practical matters" (Wikipedia).

So does it matter if (as I believe) Enterprise Architecture is not a science? The key question that raises is - what is the source and status of EA knowledge, and how can we ever resolve matters of opinion, except by obscure mediaeval argument?

SOA Retrospective

As this is my last day as a full-time employee of the CBDI Forum, I thought I'd permit myself a little retrospective discussion on Service Oriented Architecture.

My own understanding of the "service" concept can be traced back to a number of things I was doing in the 1990s, including enterprise modelling for open distributed processing (ODP), as well as early structured methods for component-based development (CBDI or CBSE). From this perspective, it wasn't hard for me to see that the service concept represented a significant departure from the software paradigms I had learned at university (everything from SIMULA to PROLOG).

By the late 1990s, the software vendors were pushing component-based development. It was already obvious to some of us that the most important thing about a component was not the internal mechanism (its implementation) but the service it delivered. I started attending meetings of the CBDI Forum, and writing for the CBDI Journal. My main focus however was not software but business - developing a business modelling approach that would link component-based software with component-based business transformation. I developed a component methodology called SCIPIO, and in 2001 I published a book on the Component-Based Business, which tried to explain business architecture in terms of business components - coherent chunks of business capability, interacting through service relationships.

But although my friends at CBDI Forum understood this, we found that a lot of people in the software industry still saw components merely as reusable lumps of software - a rehash of modular programming or object-oriented programming. What was often missing from this story was any sense of architecture. However, even at that time, a few methodologies did take architecture seriously: there were some good ideas (including explicit recognition of the service concept) in Select Perspective, for example.

By the early 2000s, people had started to talk about service-orientation and service-oriented architecture. Gradually the large software vendors got their hands on these concepts, and started to sell products and platforms into a growing market called "SOA". Lots of people started to equate SOA with these products and platforms. Meanwhile, some SOA evangelists started to make sweeping (and usually ungrounded) promises about business agility and transformation. A gap started to emerge between expectation and reality.

Some of us had experienced something similar before. In the 1980s, we had worked with Information Engineering and CASE tools, which were sold on the promise of transforming IT productivity. However, achieving these outcomes generally depended very little on the features of the technology and much more on how the technology was implemented, used and managed. I developed a technology change management roadmap, which JMA (and later Texas Instruments) used for planning the adoption of its methodology and technology into some of its larger customers.

So many years later, CBDI developed an SOA Roadmap, along similar lines, in order to help large customers adopt and exploit SOA. And in 2006, I joined CBDI as a full-time employee. One of my tasks was to help turn a considerable body of knowledge (as published over many years in the CBDI Journal) into a well-structured methodology (known as SAE - Service Architecture and Engineering).

Although my instinctive sympathies were always with the evangelists who talked about what kind of future business transformations might be possible by extrapolating the available technologies, I started to get impatient with their inability to talk concretely about business change. At the other extreme, I was underwhelmed by some of the rather narrow and uninteresting uses to which SOA technologies were sometimes being put. Regular readers of this blog will have seen me writing frequently on these subjects.

One source of growing complexity, however, is that SOA can no longer be regarded (if it ever was) as an isolated discipline. On the one hand, it tends to be combined with all sorts of other technologies and practices, including BPM, Business Intelligence, Event Processing (Complex or Otherwise), Grid, Cloud, and anything 2.o. On the other hand, it tends to be allied (sometimes awkwardly) with other architectural disciplines, notably Enterprise Architecture. And there are usually major business changes going on, such as Merger and Acquisition. Whether you are doing a technology change roadmap or a business case, you probably need to juggle several of these factors. In many situations, SOA is no longer the rising star of technology adoption but merely one of the supporting acts. David Sprott, the founder of the CBDI Forum, now sees service architecture as business as usual (BAU), saying that "the key questions to be answered revolve around the level of standardization, componentization and rationalization in business and IT" (SOA in the Recession, Feb 2009). Some SOA pundits have even suggested that SOA (or at least the label "SOA") is dead. See my post Has SOA gone for a Burton?

I have no doubt that the CBDI Forum will quietly continue applying the concepts and principles of SOA to practical business problems, unbothered by all this hype. I remain more interested in business transformation than in technology for its own sake, but I shall continue to blog here from time to time on SOA and related topics. For other topics relating to systems thinking for innovation and business survival, you may want to subscribe to my Demanding Change blog.

Deconstructing The Grammar of Business

@JohnIMM (John Owens) trots out a familiar piece of advice about data modelling today.

"Want to know what data entities your business needs? Start with the nouns in the business function names."

Starting with the nouns is a very old procedure. I can remember sitting through courses where the first exercise was to underline the nouns in a textual description of some business process. So when I started teaching data modelling, I decided to make this procedure more interesting. I took an extract from George Orwell's essay on Hop-Picking, and got the students to underline the nouns. Then we worked out what these nouns actually signified. For example, some of them were numbers and units of measure, some of them were instances, and some of them were reifications. (I'll explain shortly what I mean by reification.) Only a minority of the nouns in this passage passed muster as data entities. Another feature of the extract was that it used a lot of relatively unfamiliar terms - few of us had experience measuring things in bushels, for example - and I was able to show how this analytical technique provided a way of getting into the unfamiliar terminology of a new business area. I included this example in my first book, Pragmatic Data Analysis, published in 1984 and long out of print.

One problem with using this procedure in a training class is that it gives a false impression of what modelling is all about. Modelling is not about translating a clear written description into a clear diagrammatic structure; in the real world you don't have George Orwell doing your observation and writing up your interview notes for you.

Now let me come on to the problem of reification. The Zachman camp has started to use this word (in my view incorrectly) as an synonym of realisation - in other words, the translation and transformation of Ideas into Reality. (They can perhaps find some support for this usage in the work of the Arab philosopher ibn Arabi, who talks about entification in apparently this sense.) However, modern philosophers of language use the word "reification" to refer the elevation of abstract ideas (such as qualities) to Thingness. One of the earliest critics of reification was Ockham, who objected to the mediaeval habit of multiplying abstract ideas and reified universals; his principle of simplicity is now known as Ockham's Razor.

In our time, Quine showed how apparently innocent concepts often contained hidden reification, and my own approach to information modelling has been strongly influenced by Quine. For example, I am wary of taking "customer" as a simple concept, and prefer to deconstruct it into a bundle of bits of intentionality and behaviour and other stuff. (See my post on Customer Orientation.) As for business concepts like "competitor" or "prospect", I generally regard these these as reifications resulting from business intelligence.

Reification tends to obscure the construction processes - tempting us to fall into the fallacy of regarding the reifications as if they directly reflected some real world entities. (See my posts on Responding to Uncertainty 1 and 2.) So I like to talk about ratification as a counterbalance to reification - making the construction process explicit.

Of course, John Owens is right insofar as the grammar of the data model should match the grammar of the process model. And of course for service-oriented modelling, the grammar of the capabilities must match that of the core business services. But what is the grammar of the business itself? Merely going along with the existing nouns and verbs may leave us short of discovering the deep structural patterns.

A Value Proposition for Enterprise Architecture

#EAC2009 Something else I took away from Sally Bean’s and Peter Haine’s workshop Reflecting on EA at EAC 2009 was the relationship between two sets of questions.

• What is EA all about, what is the (emerging, changing) identity of EA?
• What is the value proposition for EA?
  

I personally find the second of these two questions much more important and interesting than the first. Questions of identity often result in entrenched positions, and (as I discussed in my previous post Think Globally, Act Locally) can produce division between different views. In the case of Enterprise Architecture, there is a traditional view (EA-as-IT-planning) and an emerging view (EA-as-business-strategy).

I think the question about the value proposition opens up a much more interesting discussion about the possible evolution and potential multi-tasking of enterprise architecture teams.

(EA itself needs to understand its business model to help it survive. The full exploration of the value proposition needs something like the Osterwalder business model canvas we discussed in our workshop on on Business Modelling for Business Improvement, so I'll have a go at that. See below for Osterwalder template.)

One of the key questions for the value proposition is the timescale. Sometimes enterprise architecture is described in terms of longer-term value: through-life coordination and capability management. Some people are still comfortable with this; but other people see a difficulty with the fact that business needs to wait so long to realise this kind of value, or even to measure it properly. In contrast, there is growing support for a much shorter-term delivery of value.

Essentially, that means EA must deliver value within same timescale as projects. And what is the nature of this value? Roger Sessions points to the massive failure rate in IT projects, and argues that's something EA can and should be fixing. In other words, the EA value proposition can be defined in terms of improving IT project success ratios.

I see two difficulties with this. The first is one of perspective. If EA is working closely with the projects, then how is the EA perspective any different from project perspective? And if the problem is that projects are doing things wrong, then how can EA fix the problem from within the project perspective? The EA view of business requirements is hardly going to be very different from that of the good business analyst on the project. If EA is no longer taking the long view, then its value proposition is largely based on the hypothesis that the architects may have a bit more knowledge and experience than the business analysts, and some slightly superior tools and techniques. But we might achieve this outcome more efficiently and effectively by simply upgrading the business analysis practice and redeploying the architects as senior business analysts. Indeed, some IT organizations seem to be moving in this direction, although they haven't taken away the formal job titles yet.

The second difficulty is that the job of overseeing projects and ensuring project success is hugely duplicated. Within a large IT organization, we might have project management, programme management, IT governance, tools and methods, quality management (control and/or assurance) as well as enterprise architecture, each with its own "body of knowledge", each trying to prevent projects from getting things wrong (and claim the credit). The word "silo" springs to mind here. (All of those roles might possibly be held by the same person, but does that remove the complexity?)

From a systems-thinking perspective, this looks completely crazy. If the value proposition for EA is simply to correct things that projects are doing wrong, then this counts as "failure demand". If the value proposition for EA is to make sure that projects are successful, then that's putting the responsibility in the wrong place. It is the project's job to be outstandingly successful. If they can achieve this unaided, this appears to make EA redundant.

In summary, I'm not convinced that the traditional value proposition for enterprise architecture is convincing to its customers, whoever they are. (Who are the customers anyway, the CIO or CFO who have to pay for it, or the business line management and IT project managers who are being asked to spend time and effort on EA "for their own good", and are not always grateful for EA attention?) I think the top priority for the enterprise architecture discipline is to find and formulate a viable and meaningful value proposition. And it really doesn't matter whether we call it "enterprise architecture" or not.


Thanks to several Twitter friends for today's discussion: A Jangbrand, Anders Østergaard, Andrew Townley, Brenda Michelson, Colin Beverage, Roger Sessions, Todd Biske. (Did I miss anyone?)

---

Business Model Template

View more OpenOffice presentations from Alexander Osterwalder.

EA: Think Globally, Act Locally

#eac2009 The IRM Enterprise Architecture conference in London this month continued some of the themes of the Open Group Enterprise Architecture conference in April, and I saw many of the same faces.

One of the main themes was the scope of Enterprise Architecture. Many people argued strongly that EA was not just about IT but about Business, and this came across from Chris Pott's keynote.

Not everyone agreed, however. When Roger Sessions challenged Chris from the floor, putting the case for the continued relevance of IT to the enterprise architect, there was a ripple of assent around the audience.

EA undoubtedly has a credibility problem if it aspires to sort out broader problems of "the business" when it is currently perceived to have had limited success with its original remit - the narrower problems of IT planning.

Roger posted the question on Twitter: "Two enterprise architecture camps: focus on the global vs. focus on the IT. Which are you?" Most of the replies took the side of the global. Nigel Green linked to something he had posted to his blog last year (What is an enterprise architect?) and said "I'm the 'global' if you mean Business Technology a la Forrester ... EA is about biz transformation not just IT". Anders Østergaard Jensen said "EA = S + B + T from which we can infer that EA is global", and Colin Wheeler said "I think Enterprise Architecture is a logical framework in which the business can make rational decisions. Definitely part of the global focus for me".

It was Tom Graves who found a way of reconciling both positions, by quoting Patrick Geddes' slogan Think Global, Act Local. "Global. IT alone is too narrow ... Lack of whole-org EA creates problems for IT. Act local (IT) think global (EA). Apply EA in detail for IT-systems etc, but always remember the global." With most of the others, Tom remains convinced about the importance of the global. "EA is architecture of enterprise, not IT - IT is just an implementation, nothing more - drop the IT-centrism!!".

Roger suggested a compromise. "You can't deliver high value IT unless you know how IT relates to the organization as a whole. The role of enterprise architecture is to bring a holistic perspective to IT." Or perhaps ""The role of EA is to bring a holistic perspective to IT-supported business capability."(miket0181)

There is clearly a wide gap between AS-IS (enterprise architects frustrated within the IT department) and TO-BE (enterprise architects respected across the business). Even if we go along with Chris Potts's line that enterprise architecture is a form of corporate strategy, there's a way to go in most organizations. Nothing wrong with thinking about the future, but some enterprise architects have got a day job as well.

EA Archetypes

#EAC2009 Following my workshop on Business Modelling for Business Improvement at EAC 2009, I caught the end of Sally Bean’s and Peter Haine’s workshop Reflecting on EA.

An interesting discussion on EA archetypes. We talked about the contrast between EA-as-visionary and EA-as-realist. The EA-as-visionary is an optimist who produces value by creating new opportunities and producing economies of scale and scope; the EA-as-realist earns his/her keep largely by stopping ill-conceived initiatives, saying No to pushy vendors, and producing economies of governance. (In January 2006, I put the case for Realism in a debate on Optimism with Jeff Schneider.)

Roger Sessions mentioned an interesting correlation in the US government space between IT failure and Sarbanes-Oxley-driven "investment" in Enterprise Architecture, suggesting that a mere formal requirement to produce EA deliverables may actually destroy value. (Roger discussed this in his recent editorial on Obama's Information Technology Priority; he is planning to include some graphs in his talk tomorrow afternoon.) This indicates a third archetype: EA-as-formalist, bureaucrats playing Zachman bingo with little vision or practical realism.

And yet there is probably a place for formal rigour, if it can be balanced with vision and realism. It is the formal rigour that confers some authority on the architect to promote either vision or realism or both. So how do we combine the three archetypes: Visionary, Realist, Formalist?

While it would be crazy for me to equate this triad with Lacan’s triad (Imaginary, Real, Symbolic), I think there may be some weak structural parallels. Here are some starting thoughts.

Imaginary - For Lacan, this is about constructing coherent images. The EA-as-visionary must be good at joined-up-thinking, and good at story-telling.

Symbolic - For Lacan, this is about representing the images in some language. The EA-as-formalist must be able to create robust representations.

Real - For Lacan, the Real is what resists representation. The EA-as-realist must understand the limitations of both the vision and the formal models.

I don't know how fruitful this parallel is going to be, so I need to think about it a bit more.

What's Wrong with Layered Service Models?

@JohanDenHaan pointed me at a couple of articles by Bill Poole

• Layered Service Models are Bad
• Layered Service Models are Bad (continued)

Bill takes a particular layered service model, a plausible combination of layers such as you might find in any number of popular SOA books, and shows how he can use this model to produce an inefficient and inflexible design.

Of course, all this shows is that there are problems with a particular layered service model, as interpreted by Bill. (The authors of the books from which Bill has taken this model might claim that Bill had misunderstood their thinking, but whose fault is that?) It doesn't show that all layered service models are bad.

As Bill points out, one reason commonly cited in support of the layered service model approach is the hope that services will be highly reusable. But there is a much more important reason for layering - based on the expectation that each layer has a different characteristic rate of change. (This principle is known as pace layering.)

When done properly, layering should make things more flexible. When done badly (as in Bill's example) then layering can have the opposite effect.

One of the problems is that the people inventing these layered service models often confuse classification with layering. We can identify lots of different types of service, therefore each type must go into a separate layer. A deeper problem with these models is that their creation is based purely on clever thinking rather than systematic and empirical comparison of alternatives. Too much architectural opinion is based on "here's a structural pattern that seems to make sense" rather than "here's a structural pattern that has been demonstrated to produce these effects".

See my post on Layering Principles.

You can't just take an SOA principle ("loose coupling is good", "layering is good"), apply it indiscriminately and expect SOA magic to occur.

Customer Orientation

@adamshostack objects to something I quoted from Clayton Christensen: Understanding your customer isn't enough (hattip Chris Lawer, Gunnar Peterson).

What Christensen is saying, and I absolutely agree with this, is that understanding the customer is the wrong level of analysis (granularity). What we need to focus on is the job the customers are trying to get done when they use your product or service.

Adam thinks this is "fascinating & wrong. W/o understanding customer orientation, you can't from job" (via Twitter).

I think pharmaceutical sales and marketing provides an excellent example of why Christensen is correct. As I have explained before, a typical twentieth-century business model involved drug company representatives making personal visits to doctors. To support this kind of model, you collected lots of information about the doctor - not just professional (size of practice, specialization, and so on) but also personal (ethnic group, sexual preference, ages of children, golf or squash). The drug company employed a range of representatives of different types, and selected the appropriate rep to visit a white gay squash-playing doctor.

The trouble with this business model is that it is not aligned with the products and services of the drug company. Doctors increasingly regard these kind of sales visits as a complete waste of time; even if they accept the hospitality of the drug companies, they have learned to be resistant to the sales messages.

What the drug company needs to focus on is how to provide more value to the doctor. For this purpose, you don't need information about the doctor, you need information about what the doctor actually does. In particular, you have to understand the decision process in which the doctor thinks about prescribing particular drugs to a particular patient, as well as the collaborative process in which the doctor and other healthcare practitioners discuss alternative courses of treatment with a patient.

Understanding the doctor is missing the point. The opportunity to create business value comes from understanding the work of the doctor. Different doctors may have different styles and habits, and may approach similar cases in different ways (although this is increasingly constrained by procedure and protocol imposed by health authorities or health insurance companies). That's the right level of analysis.

One technique we use for this analysis is business process modelling. But we're not interested in the company's own processes, we're interested in the customers' processes, and in the variations in these processes. Business survival depends on providing products and services that add value to these customer processes, so it's the customer processes we need to understand. Not the customer as a passive and indivisible entity (as in many CRM systems) but as an active bundle of behaviour and capability and purpose.

Semi-structured processes 2

@jonerp (Jon Reed) kindly sends me a link to an article on the SAP community network website, Value Scenarios and the BPM continuum, discussing the requirement for semi-structured processes.

The author of the article, one Dan Woods of Evolved Media, places "processes automated and supported by BPM" at the "loosely-structured" point of the continuum, but he also talks about "more advanced and structured process design using BPMN and other such techniques"; I don't fully understand his taxonomy of processes, but he certainly seems to understand the need to support semi-structured processes as well as the fully structured processes described by Ann Rosenberg.

Dan's starting point is the SAP book written by Ann Rosenberg and others, but his own analysis goes a lot further. He picks up on a reference to the Geoffrey Moore core/context distinction (which CBDI Forum members will be familiar with), and takes this distinction to its logical conclusion.

Dan also makes clear his opinion that SAP doesn't currently support this continuum. "SAP's integrated story about BPM, SOA, and Value Scenarios will have to be extended to include more unstructured collaboration." While Dan is not an official spokesman for SAP, the fact that opinion article is published on an SAP website does give it some credibility, don't you think?

From Business Intelligence to Organizational Intelligence

#orgintelligence ...

---

Business Intelligence (BI) is one component of what I'm calling Organizational Intelligence, so let's look at some useful trends within the BI world that are relevant to Org Intelligence, as well as some criticisms of the current state of the art in BI.

Real-time, event-driven process-driven BI

Over the past couple of years, we've seen the emergence of real-time BI platforms such as SeeWhy. Champions of realtime BI suggest that it can address weakness of conventional BI, including

• out-of-date information
• failure to identify process problems
• lack of suitability as predictors

[source: Three Weaknesses in Business Intelligence Today, CIO magazine, October 2007)

Last month Charles Nicholls (SeeWhy CEO) attacked IBM in his blog for (as he claims) neglecting the web: What about the web, IBM? At eBizQ, Michael Dortch offers a more balanced view: Some Shafts of Light. The real proof will be if IBM's new Business Analytics and Optimization Services unit will address the kind of real-time process issues identified by SeeWhy a couple of years ago.

Nari Kannan also criticizes conventional BI for its lack of connection to the business process. Drawbacks of Business Intelligence Tools When Handling Business Processes.

"When it comes to Process or Operational Intelligence, the problem becomes slightly different. Post mortem, rear view mirror kind of information is not very useful. Knowing that you screwed up a patient's surgery experience or the Automobile Insurance Claim, after the fact is not as useful as getting real-time information as and when it is happening and you have a chance to set things right."

Collaborative decision-making

In January, Gartner published some forecasts about the BI market for the next three years, including this.

In 2009, collaborative decision making will emerge as a new product category that combines social software with BI Platform capabilities. The emergence of social software presents an opportunity for savvy IT leaders to exploit the groundswell of interest in informal collaboration. Instead of promoting a formal, top-down decision-making initiative, these IT leaders will tap people's natural inclination to use social software to collaborate and make decisions.

Dave Linthicum agrees.

"This is occurring now, and is a huge area of growth in BI, if you ask me. With all of that good data being placed on the social networks these days, the BI applications around mining that data is going to be extremely valuable."

Collaborative BI

I predicted the emergence of collaborative business intelligence nearly four years ago (Service-Oriented Business Intelligence). So I am very happy to see early signs that this might now be moving into the realm of practical application. If you are a BI toolmaker or advanced user, I'd be delighted to talk to you.