Ecosystem SOA

The SOA world is finally catching up with some of the ecosystem ideas that I published in my 2001 book on the Component-Based Business (see my Slideshare presentation) and developed further in several articles and presentations for the CBDI Forum over a number of years.

• Supply and Fit (September 2000)
  
• Identifying Components 2 (October 2001)
• Opportunities in the Insurance Ecosystem (December 2004)
• Business Strategy Planning for the Service Economy (May 2006)
  

The biological approach to creating business and software services is radically different to the solution-driven approach, and is based on biological and ecological metaphors.

• First we identify an ecosystem, which may contain both human users and existing artefacts.
• Then we identify services that would be meaningful and viable in this ecosystem.
• Then we procure devices that enable the release and delivery of these services into the ecosystem.

I previously defined Three Types of Requirements Engineering, and we can map these onto different styles of SOA.

Solution-Driven (Specific)	Solution-Driven (General)	Evolution-Driven
Identify Business Problem Identify "Users" Negotiate Requirements Define Solution	Identify Domain Identify Domain Experts Define Requirements Design Solution Kit	Identify Ecosystem Identify Services Procure and Release Devices
Experimental SOA	Enterprise SOA	Ecosystem SOA

(Some people use the term Web Oriented Architecture (WOA) for what I'm calling Ecosystem SOA.)


If we regard these as phases of maturity, then we can have a straightforward roadmap from left to right, as in for example the CBDI SOA Roadmap. However, some organizations may need to tackle these styles of SOA in parallel rather than in sequence.

---

A service portfolio plan for Enterprise SOA can be based on an enterprise model that identifies the capabilities of the enterprise and clusters these into domains. In the CBDI Forum's SAE methodology, the domains are classified as Core and Contextual according to a matrix derived from Geoffrey Moore. (Note how the domains migrate around the matrix over time.) See for example my blogpost Tesco outsources core eCommerce.



A similar model, derived from Amin and Cohendet's book Architecture of Knowledge, explicitly describes Core and Periphery in terms of knowledge intensity. In other words, the reason something is classified as Core is because it encapsulates some important (strategic) knowledge.

For example, an insurance company knows more about insurance than about cars, so when providing car insurance it may decide to partner with other organizations that know more about cars than about insurance. This results in a composition of insurance-related services and car-related services (for example, determining the insurable value of a used car). Such compositions can be either directed (in other words, composed by a single dominant player) or collaborative (in other words, emerging from the interaction between multiple players within the ecosystem).

For example, an online retailer knows about her products and services, but doesn't wish to become an expert in credit card handling, or to be responsible for data protection and security, so she delegates these concerns to a specialist provider that knows more about these matters.

Similar considerations can apply to industry consortia, such as ACORD (for the insurance market). ACORD can define generic service-based assets (such as models, schemas, interfaces and so on) for insurance. However, insurance companies will also wish to use generic service-based assets to cover requirements that are not insurance-specific, such as customer management or complaints handling, where ACORD may not be able to add any knowledge-value, and it would be appropriate for ACORD to regard these as peripheral to its own activities rather than core.

---

So one approach to Ecosystem SOA is to push out from the enterprise into the ecosystem. John Hagel calls this Inside-Out Architecture, which he contrasts with Outside-In Architecture. (See my post on Outside-In Architecture.)


An Outside-In Architecture starts with a model of (the flows of) knowledge and value in the ecosystem as a whole. The strategic question for an enterprise is how to find way of both contributing value to the ecosystem, and drawing value from the ecosystem, through the provision of ecologically viable services.



For example, a telecoms company might reasonably consider that its core competence is something to do with communications. So a positional strategy would drive it to a dominant position in the middle of a large communication ecosystem, providing a platform of services that add value to a diverse range of communication activities by other people. The dominant position would allow it to negotiate a strong share of the value generated.

However, respect for the ecosystem would lead it to leave sufficient value to third parties to maintain the economic health of the remainder of the ecosystem. Instead of a simple positional strategy, a relational strategy (based on mutual trust with ecosystem partners) should produce a more sustainable and ecologically sound ecosystem.

---

Service Ecosystem from Richard Veryard


Related post: Ecosystem SOA 2 (June 2010)

Is Enterprise Architecture a Profession?

The Center for Advancement of Enterprise Architecture Profession (CAEAP) is in the process of developing an EA Professional Practice Guide that, among other things, will define what it mean to be an Enterprise Architect. Is Enterprise Architecture a profession, or does it have any reasonable chance of becoming a profession in the foreseeable future?

While I don't fully agree with the Wikipedia definition of Profession, I think it identifies a lot of characteristics commonly associated with professional status, and EA lacks many of these characteristics.

Here are three reasons why I don't think EA is (yet) a profession.

1. Organizations claiming to represent enterprise architects have relatively few members.

2. There are no meaningful sanctions against maverick or incompetent practitioners.

3. Although some universities are starting to offer degrees in Enterprise Architecture, for the foreseeable future this qualification will only be held by a tiny fraction of practitioners, mostly at the inexperienced end.

There are some worthy goals on the CAEAP homepage, which I think accord with my view of what counts as a profession, but these are currently merely future aspirations. I acknowledge that CAEAP seems to be putting a fair amount of effort into this, but this doesn't justify some of its more extreme supporters trying to confuse AS-IS with TO-BE. I shall be prepared to regard EA as a profession and the CAEAP as a legitimate representative body if and when these goals are ever achieved. For the time being, I think it is more accurate to regard EA as an aspiring profession than an established one.

However, the present interest in CAEAP is almost entirely coming from enterprise architects themselves, rather than from the people they are providing services to, and I think this may be a major obstacle in the CAEAP's achieving its goals.

As I said in my previous post (Is Enterprise Architecture Dead?), the desire for professional status is not coming from the demand-side (CEOs wishing to distinguish genuine practitioners from charlatans) but from the supply-side (like teenagers wanting to be taken seriously). People who think EA is a waste of space are not going to be reassured by the existence of a cartel of people with expensive but vacuous certificates. Meanwhile the most experienced and able practitioners are getting on with the work, engaging with the business rather than worrying about preserving a label with such negative connotations.

---

Another major issue, in my view, is that of professional accountability. In healthcare, professional and ethical responsibility is clearly focused on the patient (as the "consumer" of healthcare services). In the legal profession, a lawyer represents a specific client and there are clear conflict-of-interest rules preventing a lawyer representing multiple parties in the same case. The CAEAP Value Map recognizes the need for "professional accountability" as well as "responsibility to multiple stakeholders and seek(ing) balance among potentially conflicting demands". But to which of these multiple stakeholders is the Enterprise Architect ultimately accountable, and who has the ultimate authority to resolve conflict?

I believe that CAEAP should be talking not just to EA practitioners but also to the consumers of EA services - whoever they might be - and I don't see any sign that they are doing this. (However, they are happy to accept sponsorship from vendors, who clearly have a commercial interest in influencing enterprise architects. This kind of sponsorship is perfectly acceptable for a trade body, but raises questions for an organization that wishes to establish itself as a proper profession. The Royal College of Midwives doesn't take backhanders from drug companies, does it?)


CLARIFICATION: Microsoft and IBM sponsored the CAEAP summit. @jpmorgenthal insists that this is a conference, distinct from the organization itself, and argues that the organization itself is not sponsored by vendors. That may be strictly true, but it is difficult to avoid the impression that sponsoring the conference significantly benefits the organization.

Jon Ayre challenges this point. "Consumers choose whether or not to consume. If EA doesn't provide right services consumer will reject it."

Of course, with some exceptions, professionals cannot force their services upon their clients. But there is still some external validation - a practice becomes a profession not because people inside the practice want it to be, but because people outside the profession have respect for it. Members of a profession may receive public or institutional funding for some of their activities, and may have a privileged status in legislation and regulation.

As far as I can see, all of CAEAP's directors and trustees are EA insiders. So where is the external voice? Who represents the stakeholders of EA?

Is Enterprise Architecture Dead?

I can't see much recognition or respect for the value of Enterprise Architecture except among the ranks of EA practitioners. In many organizations, the function of enterprise architecture is squeezed or marginalized, if not rejected altogether.

The attempt by the CAEAP to turn EA into a Profession is not going to address this problem. The desire for professional status is not coming from the demand-side (CEOs wishing to distinguish genuine practitioners from charlatans) but from the supply-side (like teenagers wanting to be taken seriously). People who think EA is a waste of space are not going to be reassured by the existence of a cartel of people with impressive-looking certificates.

Meanwhile the most experienced and able practitioners are getting on with the work, engaging with the business rather than worrying about preserving a label with such negative connotations.

---

I have been mulling on this post for a while, but I am now prompted to complete it by CBDI boss David Sprott, who has just produced a good article on The Death of Enterprise Architecture. Perhaps responding to the fuss about the Death of SOA, which exercised a few minds earlier in the year (see my post Has SOA Gone for a Burton?), he suggests that it is Enterprise Architecture that is dead - not just in need of a new label but in need of a new concept.

David proposes Smart Ecosystem Architecture. He refers to some of the CBDI reports I wrote about ecosystems in various industries (Airlines, Insurance, Public Sector) and argues that it's not about the enterprise any more.

Various influences particularly Complex Event Driven Architecture and Smart Business and IT are strongly predicated on optimizing business design and processes involving all the ecosystem stakeholders.

Not just engaging with the business, then, but looking beyond the business into the demand environment. See my paper (with Philip Boxer) Taking Governance to the Edge (Microsoft Architecture Journal, August 2006)

See also Has EA gone for a Burton? (August 2009), Is Enterprise Architecture a profession? (October 2009), Emergent Architecture (March 2011), Towards Next Practice EA (May 2013)

Event-Driven Enterprise Architecture

@toddbiske responded to @jeffrschneider 's question about the trigger for EA services.

• What are your EA Services?
• EA Services: Part Two

 Todd distinguishes between two kinds of service, which he calls request/response-style and event-driven services.

"There are services that are explicitly invoked at the request of a consumer, and then there are services that are executed in response to some event. In the latter case, the team providing the service is the one monitoring for the event. If some other team was monitoring for it, and then told your team to do something, then we’re back to the request/response style with one team acting as consumer and the other acting as the provider."

But surely a request is an event? The EA team may have many conflicting demands on its time and attention, and so there is always a decision (perhaps driven by some policy) whether and how quickly to respond to a given request.

A kind of event that might trigger EA services is one that looks like "we think we might have a problem in such-and-such area". (This is what Philip Boxer calls a P-type service - rcKP - services at the edge.)

In this example, "we" could be the EA team or the EA customer or both together. The service might include clarifying whether there really is a problem at all, not just solving a known problem.

The event triggering this kind of work is typically a weak signal. There are usually more possible problems than we actually have time to work on, so the existence of a problem is not a sufficient trigger for activity.

One interesting question here is how the event-driven EA actually works. The EA team must have some view of a complex and dynamic world to which it is attempting to add value, that allows it to organize its work. In other words, it needs a model (possibly implicit, but preferably explicit). But what model is this? Not just the enterprise model itself, not just the enterprise architecture framework (TOGAF or what have you) but something else as well.

Towards an Architecture of Privacy

@futureidentity (Robin Wilton) posted some interesting ideas about Identity versus attributes on his blog.

"For an awful lot of service access decisions, it's not actually important to know who the service requester is - it's usually just important to know some particular thing about them. Here are a couple of examples:

• If someone wants to buy a drink in a bar, it's not important who they are, what's important is whether they are of legal age;
• If someone needs a blood transfusion, it's more important to know their blood type than their identity."

However, there is an important difference between Robin's two examples. Blood transfusion is a transaction with longer-lasting consequences. If a batch of blood is contaminated, there seems to be is a legitimate regulatory requirement to trace forwards (who received this blood) and backwards (who donated this blood), in order to limit the consequences of this contamination event and to prevent further occurrences.

There is a strong demand for increasing traceability. In manufacturing, we want to trace every manufactured item to a specific batch, and associate each batch with specific raw materials and employees. In food production, we want to trace every portion back to the farm, so that salmonella outbreaks can be blamed on the farmer. See Information Sharing and Joined-Up Services 1, 2.

Transactions that were previously regarded as isolated ones are now increasingly joined-up. The eggs that go into the custard tart you buy in the works canteen used to be anonymous, but in future they won't be. See Labelling as Service 1, 2.

There is also a strong demand for increased auditability. So it is not enough for the barman to check the drinker's age, the barman must keep a permanent record of having diligently carried out the check. It is apparently not enough for the hotel or bank clerk to look at my passport, they must retain a photocopy of my passport in order to remove any suspicion of collusion. (The bank not only mistrusts its customers, it also mistrusts its employees.)

There is a large (and growing) class of situations where so-called joined-up-thinking seems to require the negation of privacy. I am certainly not saying that this reasoning should always trump the needs of privacy. But privacy campaigners need to understand that all transactions belong within some system of systems, and that this provides the context for the forces they are battling against, rather than pretending that transactions can be regarded as purely isolated events. The point is that authorization is not an isolated event, but is embedded in a larger system, and it is this larger system that apparently requires greater disclosure and retention.

@j4ngis asks how long chains to use for traceability. What "length" of traceability is sound and meaningful? How do we connect all these traces? And also backward and forward in the "chain". For how long should records be kept?

• Should we also know the batch number for the food that was given to the chicken that laid the egg you included in the cake?
• Do we have to know the identity of the blood donor after six months? 10 years? 100 years?

The trouble is that there is no rational basis for drawing the line. It is always possible that some contamination in the chicken feed might affect the eggs and thereby the custard tart. It is always possible that the hyperactivity of certain schoolchildren, or the testosterone levels of certain adults, might be traced back to some contamination in the food chain. It is always possible that some obscure data correlation might one day save lives or protect children. And given the vanishing costs of data management, even a faint possibility of future benefit appears to provide sufficient reason for collecting and storing the data.

Robin clearly supposes that attribute-based authorization is a "Good Thing". I am sympathetic to this view, but I don't know how this view can stand up against the kind of sustained attack from a certain flavour of joined-up systems thinking that can almost always postulate the possibility (however faint) of saving lives or protecting children or catching criminals, if only we can retain everything and trace everything.

For my part, I have a vague desire for anonymity and privacy, a vague sense of the harm that might come to me as a result of breaches to my privacy, and a surge of annoyance when I am required to provide all sorts of personal data for what I see as unreasonable purposes, but I cannot base an architecture on any of these feelings.

Traditional arguments for data protection may seem to be merely rearguard resistance to integrated and joined-up systems. Traditional architectures for data protection look increasingly obsolete. But what alternatives are there?

---

Update May 2016

Traceability requirements for Human Blood and Blood Components are specified in Directive 2005/61/EC of the European Parliament and of the Council 30 September 2005 (pdf - 63KB)

Robin's point was that blood type was more important than identity, and of course this is true. Donor and recipient identity must be retained for 30 years, but that doesn't mean sharing this information with everybody in the blood supply chain.

Asymmetric Demand for Defence Equipment

An independent review into the way the MOD buys equipment for Britain's Armed Forces has been published today, Thursday 15 October 2009. [Report, MoD News Article, BBC News]. Key finding.

"The Ministry of Defence has a substantially overheated equipment programme, with too many types of equipment being ordered for too large a range of tasks at too high a specification. This programme is unaffordable on any likely projection of future budgets."

That situation might sound familiar to a lot of managers, not just in the defence sector.

The report makes some favourable comments about the Through Life Capability Management (TLCM) programme, but indicates a lack of hard financial data that would be required to make quantitative decisions. There has been some discussion along these lines published in the RUSI Journal, including Agility and Innovation in Acquistion (Feb 2008) and The Meaning of Value-for-Money (Feb 2009).

The explanation for the current crisis can be found in the essential multi-sidedness of the defence acquisition ecosystem. Traditional cost accounting approaches (such as activity-based costing) fail to address the complexity of this multi-sidedness, and researchers are urgently seeking alternative cost accounting methods appropriate for complex systems-of-systems.

One of the key issues for Through Life Capability Management is that any errors or omissions in the long-term equipment programme must be repaired through what are known as Urgent Operational Requirements (UOR), which over the long haul can prove far more expensive and inflexible than the planned equipment.

The report also praises the Smart Acquisition programme, and expresses regret that the disciplines of Smart Acquisition have been somewhat diluted by recent reorganization.

---

Is this report only relevant to the defence sector, or can other sectors glean anything useful? My view is that the complexities of multi-sided markets and asymmetric demand can be found in many, perhaps most sectors. And the question of coordinating effectively between short-term and longer-term spending can be found in many domains, notably IT. I have little doubt that whatever management tools and techniques are developed by the MoD and its partners to address this problem will eventually trickle into civilian management.

Online pricing practices to be regulated?

The British Office of Fair Trading (OFT) is starting an investigation of online pricing practices. [BBC News, 15 October 2009]

Type	Description	Typical Offenders
Drip pricing	Consumers only see an element of the price upfront and end up paying much more due to optional or compulsory extras	Airlines, car hire firms and insurance companies
Time-limited offers	For example, sales that finish at the end of the month or last for one day only.	Carpet stores and furniture sellers
Baiting sales	A company advertises discounts to attract visitors whilst having few items at that price on sale
Reference prices	Artificially inflating the pre-sale price of an item in order to make the discount look more attractive.	Companies offering cruises or selling furniture. Supermarkets
Complex pricing	It is difficult for a consumer to assess an individual price, such as with three-for-two offers and 'free' add-ons.	Mobile phone companies, supermarkets and computer stores
Surge pricing	Prices increase dramatically in response to peak demand	Uber
Customized pricing	Prices are individually tailored using information collected about a consumer's internet use
Price comparison websites	There may be some hidden financial ties or other collusion between an apparently independent comparison site and the suppliers whose prices they are comparing

I have talked about some of these pricing schemes and scams before, in particular complexity-based pricing. I've also talked about ways (such as the infamous Ghetto Latte) whereby consumers can get a better deal than the service provider intended. All this kind of thing results in chronic distrust between service provider and service user, and excessive transaction costs. Might seem like a classic argument for regulatory intervention, if we could really believe that would make the situation fairer. What do you think?

---

Update: Surge Pricing added 25 September 2017
NYE Surge Pricing Explained (Uber Website December 2011, via Web Archive)

Disruptive business model for telephony

@martingeddes believes that "voice telephony is a market full of potential for the service providers, applications developers and innovators, and full of promise for both end customers and upstream business organisations such as contact centre operations" (New business model aims to bring voice back, BT plc Innovation).

"What would really be disruptive is the complete turning upside down of the telephony business model."

Martin identifies three key challenges, which roughly correspond to the three asymmetries identified by Philip Boxer and myself in Metropolis and SOA Governance (Microsoft Architecture Journal, July 2005).

Martin's first challenge is how users connect. “In the case of contact centres, traditional outbound calling can be unproductive for agents with an average of around fifty per cent of an agent’s time being spent on useful calls.”	The third asymmetry requires separating out the different contexts of use
Martin's second challenge is how upstream business and end-users interact. “This is where we would need to integrate our understanding of the customer with what the upstream organisation wants to do to interact with them. Small, network and applications-based intelligent telephony improvements that enable this to happen could lead to profitable interactions for businesses and better experiences for end-customers.”	The second asymmetry requires separating out business models that can organize supply from the solutions that are on offer.
Martin’s final challenge relates to how transactions will work. “The key here is developing and implementing intelligent telephony services.”	The first asymmetry involves separating out technology from the supply of specific products.

To become better at capturing asymmetric forms of demand, an organization such as BT needs leadership that will enable it to do two things:

• Take power to the edge of the organization: The people at the edge of the organization with the relationship to the asymmetric demand must be able to organize the business model they need to capture that demand. 
  

• Develop an agile infrastructure: providing business services that can be orchestrated and composed at the edge in response to the particular forms of demand they are targeting. This then allows the supply-side of a business to extract economies of scale or scope when providing support across multiple business models.