Mollusc-Driven Architecture

Thanks to Brenda Michelson, I have picked up a wonderful example of Smart SOA from Sandy Carter of IBM.

"RFID tagging blue mussels to identify oil leaks -- mussels closes, alert sent, line stopped; no mussels harmed"

According to Steve Núñez of Illation, this is done by Norwegian oil company StatOil. See Benefits of Going Green: Who Would Have Thought? I hope the alert is not sent by snailmail.

(Aside: For their role in detecting leaks, blue mussels have been dubbed the canaries of the sea. And don't miss the Statoil recipe for Manhatten Mussel Chowder!)

For a longer description, plus links to Sandy Carter's presentation, see Brenda's post Sandy Carter on Smart SOA in Tough Economic Climate.

Straight-Through Processing 3

Where have all the agents gone? asks Seth Godin. He is talking about travel agents, stock brokers, real estate, and so on.

"The problem with being a helpful, efficient but largely anonymous middleman is pretty obvious. Someone can come along who is cheaper, faster and more efficient. And that someone might be the customer aided by a computer."

Straight-through processing - sometimes seen as one of the potential sources of value from service-oriented architecture - is a mixed blessing, as I've pointed out before. There is always a danger of disintermediation, especially if the added value provided by the middleman appears trivial or easy to replicate. As Seth Godin points out, if you are just providing an anonymous human approximation of Google, don't bother - Google does it better.

In some industries, the middleman had become lazy - taking a cut for not doing very much. My local travel agent never gave me any useful information or advice, merely handed out brochures from the travel companies and struggled with the complexities of the booking system; so I stopped using them. And in the financial markets there are still companies that think they deserve a percentage of your pension fund in return for pressing a few buttons now and again.

In the past companies like these have often been able to take advantage of a strategic position in some larger process, extracting "rent" but without creating value. And there are doubtless still many niches in the system ecosystem that can be exploited in this way.

However, service ecosystem niches that allow companies to draw rent without creating value are going to be short-lived, and rightly so. In hard times, the percentage should only go to companies that are providing genuine value. (Actually the same principle applies all the time, but people are more motivated to follow this principle now than they have ever been before.) And ecosystem SOA should help us to make sure of this.

Then business strategy in a dynamic service ecosystem should not be based on finding and maintaining positional advantage, but on creating and maintaining genuinely productive relationships.

---

Previous posts

• Straight-through processing (Feb 2005)
• Straight-through processing 2 (April 2005)

Enterprise as a Network of Events

From the early days of SOA, we've talked about understanding the enterprise as a network of services, but clearly this is not the only possible viewpoint. Can we understand the enterprise as a network of events?

In a recent post SOA and Events - Two Sides of BAM?, Ramesh Loganathan (Progress) explained how a process view of the enterprise could be flipped into an event-oriented view. Ramesh's example is in governance risk and compliance (GRC), but it looks as if the same idea could be applied to a much broader range of business requirements.

Meanwhile, in Decision Management and software development, James Taylor wants to model the enterprise as a series of decisions. But of course a decision can be understood as a special kind of event. See Model Driven Engineering and Event Processing by Opher Etzion (IBM). See also SOA is dead; long live Model-Driven SOA by Johan den Haan.

In his post Event, Rules, Processes, Decisions, Paul Vincent (TIBCO) identifies an important gap in the notations (BPMN, BMM) available for modelling these aspects of the enterprise. He talks about decisions in terms of rules, but I presume he is associating the rule with the decision process (this is how we authorize transactions) rather than with the specific instance of decision (I hereby authorize this transaction).

At the strategic level, we need to understand the relationship between an external set of possible events and an internal set of possible events. Each enterprise is capable of responding in certain ways to certain classes of external events. Making an enterprise more agile means enabling the enterprise to mobilize more appropriate responses to a greater range of external events, without proliferation of unnecessary complexity. In system thinking this is called Requisite Variety.

So if we think about the enterprise as a network of events, this gives us a direct way to reason about strategic business improvement. But what are the CEP vendors doing in this space? I can see a lot of talk about specific event-driven applications, but where is the architectural thinking about the event-driven enterprise?


Update: After I posted this, Nigel Green drew my attention to something he wrote a couple of years ago on the case for a clear distinction between events and content, which he describes in terms of the wave-particle metaphor.

Business Capability Modelling

Picked up some fragments of a discussion between Colin Jack and Udi Dahan (Twitter, March 3rd-4th).

Colin Jack: Somehow our attempts at business capability modelling ended up producing a list of entities (Customer/Order). Meh.

Udi Dahan: When doing BC modeling, focus more on use cases

Colin Jack: I think I see what you mea, but I still think complexity cost of BC's is rel high so sometimes modules are enough

Udi Dahan: Complexity of BCs isn't that high - simple, straightforward pub/sub

I may be missing some of the pieces and context that might make these fragments more intelligible, but I'm going to comment anyway, on what I imagine they might be talking about.


A key challenge of any kind of business modelling or system model is to decompose into things that make sense. For many analysts, the things that make most sense are ... things. Objects. Entities. Assets.

If these things are important to the business, they need to be looked after. So when Colin identifies Customer and Order as business capabilities, he presumably means customer-management and order-management. The customer-management capability is typically going to be responsible for discovering and managing all aspects of all instances of CUSTOMER; and the order-management similarly for all instances of ORDER.

(Gunnar Peterson adopts a similar approach for security requirements - understanding security as the protection of a range of assets. Thus security capabilities are all asset-related. See my discussion Business Model Drives Security.)

Is there anything wrong with this as a capability model? Well, it tells us what needs to be managed or protected, but doesn't exactly tell us what needs to be done.

Udi offers a radically different approach: look at the process model. Not the whole processes, but broken down into Use Cases. What we'd expect from this approach is a much larger number of much smaller and simpler capabilities, and this is confirmed by the exchange between Colin and Udi.

When I do business capability model, I want to identify capabilities with certain characteristics - meaningful and self-contained chunks, with high cohesion and low coupling. Generally this means looking at both the entities and the process. In a complex situation it is useful to draw a matrix of the interactions between entities and processes, in order to work out the clusters.

There is growing interest in capability modelling and capability management, including through-life capability management, and the capability model is a powerful tool for thinking strategically about the loosely-coupled enterprise, so I guess I shall need to write some more on this topic soon.

Pay by Results - Service Efficiency

Redmonk's James Governor reports from a Logica briefing

"nice! UK Medical Research Council pays more money to its call center supplier Logica the fewer calls it takes! good service = fewer calls!"

In other words, service payment increases as service utilization goes down. This is a very interesting example - somewhere between output-based pricing and value-based pricing - and may well be deliberately based on John Seddon's concept of "Failure Demand".

Seddon's concept is based on the observation is that there are two reasons why a service is invoked.

• Sometimes because the consumer actually wants something of direct value - Seddon calls this Value Demand.

• But more often because something has gone wrong somewhere, and the consumer wants it fixed. Seddon calls the Failure Demand, and observes that much of the activity of service-based organizations (especially call centres) is coping with Failure Demand.
  

Clearly the answer is not to design services so that they can handle Failure Demand more efficiently and cheaply. The answer isn't even to automate the Failure Demand, so that the User can get a status report without talking to a real human being. The answer is to design joined-up systems that don't generate so much Failure Demand in the first place.

A number of management consultants/consultancies have latched onto this concept.

• Dave Droar (Arrod)
• JSK Solutions
• Sharp-End Training
• Richard Oliver (Purposive Drift)

And here is the concept From the Horse's Mouth (via Benjamin Mitchell)


What does this concept mean for the design of service-oriented systems? One thing I draw from this is that it provides yet another argument why you shouldn't start the design from a set of use cases. And I am also interested in the implications for service pricing and governance of the arrangement between MRC and Logica. If you know of any similar examples, please comment below or contact me.

What's wrong with Enterprise Architecture?

Is Enterprise Architecture an Oxymoron, as David Riddell McGhee (Microsoft Australia) seems to think (via Florian Krueger)?

It would only count as an oxymoron if the word "enterprise" was fundamentally incompatible with the word "architecture". There are many possible criticisms of enterprise architecture, and I've articulated some of them myself, but "oxymoron" isn't a valid one.

If you go to Wikipedia: Figure of Speech, as I did, you will find a long list of better words. I had never heard most of them, but I picked out a few that looked as if they might apply to "Enterprise Architecture", including Auxesis (a form of hyperbole, in which a more important sounding word is used in place of a more descriptive term) and Metalepsis (referring to something through reference to another thing to which it is remotely related).

Do words matter? Many enterprise architects think it matters whether we use the right concepts; many non-architects think this idea is pedantic time-wasting. You choose.

---

Meanwhile, a more serious criticism of EA has been published on the British Computer Society (BCS) website. Enterprise architecture - dogmatic and over-ambitious? by Peter Kemp and Dr John McManus, January 2009. They identify the following characteristic problems.

Technology-led	standardisation of applications, systems and technologies used as driver for enforced business change
Dogmatic	nominal standardisation at an enterprise level is seen as a more important goal than meeting end users' real requirements
Over-ambitious	few EA strategies seem to be able to stop short of a idealised, perfect scenario
Unverified	no one has properly analysed the achievability or sustainability of the proposed EA
Divorced from the current state	although the current state is usually shown, there is no analysis of how the first steps can be taken from the current to the idealised future state
Futuristic	the EA strategy plans so far in advance that it doesn't sensibly guide the immediate next IT strategy steps
Politicised	things are reduced to sound-bites and perceptions and not judged on hard analysis of benefits and weaknesses

---

Brian Burke (Gartner) "Are there problems with some EA programs? Of course, but that doesn’t mean that EA should be abandoned and we should retreat to the failed practices of the past."

Mike Rosen (Cutter) "Don’t give up on the idea of EA, even if your past experience has been painful."

As Sally Bean comments though, some more concrete examples or stats would have been nice. For my part, I am generally suspicious of the idea that if something isn't working, the answer is to do more.

Tesco outsources core eCommerce

In March 2009, Nick Lansley explained the background to Tesco.com adopting ATG e-commerce platform.

Here are some of the key points.

1. For years Tesco differentiated itself from the competition by writing its own e-commerce software that fitted in perfectly with its processes.

2. Competitors who tried to copy this software were unable to replicate its internal functionality. (Nick refers to "special algorithms".)

3. But the "core systems" are not Tesco's "core business". So Tesco has decided to outsource.

4. Tesco selected an e-commerce software platform that matches its core systems the closest.

5. With the core already there, Tesco will devote its IT staff to designing and developing new business improvements and ideas.

---

Analysis

This story fits with the view of "services on the fault line", which the CBDI Forum adopted and adapted from Geoffrey Moore. Capabilities migrate around the grid. The core e-commerce platform is migrating from top-left to bottom-right; meanwhile Tesco is devoting its internal IT resources to generating new functionality from bottom-left.



The classification of capabilities depends partly on the link to business outcomes / value. It also depends on the knowledge associated with each capability. Correct decoupling between capabilities depends on encapsulating the knowledge (know-how) embedded in each capability. (This is a version of the well-known architectural principle: separation of concerns.)

So we can understand the strategic cycle of capabilities (core and contextual) proposed by Geoffrey Moore, by reference to a model of the knowledge cycle proposed by Max Boisot.



I don't know whether the "special algorithms" remain proprietary to Tesco, or whether they (or at least the "commodity" provided by these algorithms) become available to other users of the same platform. In any case, I guess Tesco will be looking to develop new and more sophisticated algorithms.

---

See also

CBDI Forum, SOA Fundamentals (2009) - key extract from page 222

Richard Veryard and Philip Boxer, Metropolis and SOA Governance (Microsoft Architecture Journal 5, July 2005). Philip Boxer and Richard Veryard, Taking Governance to the Edge (Microsoft Architecture Journal 6, August 2006)

Richard Veryard, Business Strategy Planning for the Service Economy (CBDI Journal May 2006)

In his post Managing over the whole Governance Cycle (April 2006), Philip Boxer extends the Boisot model to produce a governance cycle appropriate for the platform-based business. See also my post on Knowledge and Culture (April 2006).

Related posts: Service Planning (December 2006), Ecosystem SOA (October 2009) Ecosystem SOA 2 (June 2010)

Updated 13 October 2015. Links updated 18 November 2019

Business Strategy and Alignment

Which relationships dominate an organization? Some organizations are driven by customer relationships, some by partner/supplier relationships, and some by technology and research.

One of the ideas I picked up many years ago from a paper by Professor Joseph Tidd was the strategic importance of this choice: which external relationships are dominant, and how this affects the internal power relationships within the organization.

• For example, a technology breakthrough strategy would be driven by R&D, often in close collaboration with companies that would normally be regarded as competitors.

• By contrast, a strategy of technology fusion would require a much stronger role for production, with close links to suppliers of component technology.

Obviously such business strategies will need to be supported by information systems that communicate across and between organizations in an appropriate manner. Some strategies may require so-called Chinese walls, providing a level of protection from information leaking prematurely to competitors. Some strategies require a degree of proximity bordering on intimacy - business jargon refers to this as "getting into bed with" your suppliers or customers or business partners.

So I was interested to read an interview with Prith Banerjee, director of HP Labs (Riding the Recession the HP Way, BBC News, 14 March 2009). Here are some key quotes.

The world's largest technology company says a major reorganisation of research efforts last year will help it survive the downturn and secure its future. In 2008 HP announced a "groundbreaking" move to align the work done in its labs more closely with business goals. ... While most companies keep their most valuable research projects under wraps, HP has taken a different tack. Everything is out in the open and there is a real emphasis on collaboration with universities, government and other industry players.

In other words, HP believes the business architecture implemented last year gives it greater viability in the current environment. We shall see if they are right.

---

Tidd, J. 'Technological Innovation, Organizational Linkages and Strategic Degrees of Freedom', Technology Analysis and Strategic Management 5(3) 1993, pp 273-284

SaaS for the Pessimists

One of the critical success factors of SaaS is the paradox of confidence.

As I've pointed out here before, SaaS (Software-as-a-Service) helps you to manage your Service-Oriented Cashflow. By shifting from fixed costs upfront to variable costs later, you may be getting a cashflow advantage even if you end up paying more in total.

Now here's the thing about confidence. Optimists prefer fixed costs because they expect to get economies of scale; pessimists prefer variable costs because they fear they might not. So in an economic downturn, it makes sense for pessimists to shift a lot of expenditure to Pay-As-You-Go.

Some analysts are comparing SaaS (Software-as-a-Service) with ASP (application service provision). Although ASP had some notable successes, it wasn't anything like as commercially successful as its champions hoped. However, in times of credit crunch and pessimism, the cashflow arguments may be much more telling this time around.

Now here's the second thing about confidence. Pay-As-You-Go may improve the cashflow for the service consumer, but it typically causes cashflow problems for the service provider. So maybe only the most confident service providers - with the best technology and the most confident investors - can afford to offer services on a Pay-As-You-Go basis.

And here's the third thing about confidence. The consumer needs to be confident that the service provider is going to stick around. That means financial stability as well as technical stability. Does this mean it's the big companies who are going to make all the money from SaaS?

Microsoft certainly hopes that "Customers will write us bigger checks". See comment by Ed Brill (IBM), via Esther Schindler. But Microsoft can probably afford to wait for the cheques to roll in. Smaller companies will somehow have to find a way to finesse the cashflow.

Service-Orientation and Bureaucracy

A lot of business services are difficult to value. Take HR for example.

During an economic crisis, HR performs some pretty unpopular services. On the one hand, they may be the bearer of bad news to the redundant employee. On the other hand, they may restrict the authority of the line manager to dismiss employees without following the proper procedure. Does HR bureaucracy get in your way? asks Michael D Haberman. Newest despised role, suggests Dana Gardner, they are in no-win situation given economic climate.

However, the unpopularity of HR is nothing new. I fished out an old article called Taking on the Last Bureaucracy (Fortune Magazine, January 1996) and showed it to a few people, including Annrai O'Toole.

As ESB afficionados may recall, Annrai was the boss of Cape Clear Software, which was acquired by SaaS company Workday last year. Workday offers HR-as-a-Service plus Payroll, Worker Spend Management, Financials: ERP-as-a-Service, with a strong focus on human capital management.

Annrai agreed that the article was (sadly) still relevant, and made three further points

• HR must be strategic (not paper pushing) ...
• performance measurement must be automated and linked to business performance, not just subjective objectives - better BI (on hard business metrics) is the way to go.
• HR tooling must breakdown the barrier between the workers it serves and the environment they work in

Okay, let's suppose that most of the paper-pushing can be delegated to automated services (whether inhouse or SaaS). And let's suppose that performance measurement and policy compliance can be automated and made available for business intelligence services. And let's suppose that we have a decent set of communication and social networking tools.

But what is the business model for HR? What value does HR provide at the business level, and how can we design a portfolio of services to maintain and enhance this business value? If HR is strategic, does this mean that each enterprise has different requirements for HR services?

Bureaucracies are relatively easy to automate, if that's what you want to do. You can make a bureaucracy faster and more efficient, and you can move a lot of clerical processing onto an online self-service portal. But making a system more efficient doesn't necessarily make it more effective. So good design requires system thinking at the business level, before you dive into computer systems architectures, service-oriented or otherwise.

Maybe I'm being old-fashioned, but I can't see that it makes much sense to invest in an SOA or SaaS project to reduce the headcount of the HR department by (say) 25% if you still don't have much idea what value the remaining 75% are going to provide, and how you are going to make them more effective.

Modelling Identity and Context

Language is Strange says Don Ferguson. He's talking about the curious difference in Spanish between ser and estar.

• ser is used for permanent characteristics (identity). For example, Don is American, I am British.

• estar is used for transient characteristics (context). I don't know exactly where Don is at the moment, but I'm in London. Don is now working for CA; he was working for Microsoft, and IBM before that.
  

This is an important distinction to make when designing information systems and information services. It affects the way databases and database keys can be designed (if it can change you shouldn't use it as a permanent identifier) and it affects the set of operations that have to be designed (if it can change, you have to be able to change it).

The question is: how do you decide whether something is permanent or transient. As Don points out, some of the distinctions in Spanish may seem counter-intuitive to an English-speaker. Location is always assumed to be transient, even for fairly solid things like the American Embassy, while origin (as in "I'm from Massachusetts") is assumed to be permanent.

When you are modelling a large complex enterprise, you really don't want to be working this kind of thing out on a case-by-case basis. So it is generally good enough to adopt some broad patterns (e.g. LOCATION is always going to be transient) even if it occasionally looks wrong. (It's usually better to err on the side of allowing something that isn't always going to be needed, rather than banning something someone might want.)

The "good enough" strategy is known as "satisficing". It may not be as perfect as full optimization, but it's a lot quicker. Clearly Spanish is good enough.

Where has all the SOA gone?

What on earth did the banks do with all the money?

No, I'm not talking about the money they blew on dodgy financial instruments. I'm not talking about Sir Fred Goodwin's pension. I mean all the money they spent on SOA, CEP, and other leading edge software.

When I first read Joe McKendrick's headline (Analyst: SOA may have reduced severity of mortgage crisis), I thought he meant that the crisis would have been even worse if the banks hadn't been doing SOA.

But when I read the piece, I found that he meant the exact opposite - that the crisis might not have been so bad if the banks had done SOA properly. This is based on a recent statement by James Governor, at a presentation hosted by IBM (Virtual Global SOA Forum),

"if we'd invested more in integration over the last few years, and better data governance, we would have significantly reduced the risk in terms of our banking infrastructure, our credit infrastructure, the kinds of mortgages that went out there"

In other words, despite countless SOA projects across the finance sector, the banks have failed to deliver adequate integration and data governance.

People with incredibly long memories may remember IBM's claim that "the banking and insurance industries lead in the maturity of their SOA deployments" [IBM Flatters Finance Sector (June 2008)]. Or that it was a bank that won the SOA Consortium Case Study Contest [More Flattery for the Finance Sector (September 2008)].

I don't think anyone is saying that these SOA deployments could have saved the world economy from crisis. But couldn't smarter SOA have made a difference to the outcome? With hindsight, how clever were all those SOA deployments, really? How can a bank have SOA maturity if it isn't dealing with integration and data governance?

I agree with James that SOA should make a significant contribution to restoring the operational viability of the banks. But I don't agree that the right answer is just more investment in SOA. I think what is needed is more intelligent investment, sensibly planned to deliver lasting business benefit from value-for-money SOA. That's what real SOA maturity means.