Complexity and Power 2

#entarch #orgintelligence In an earlier post, I pointed out the architectural trade-off between Complexity and Power. There may be a choice between a simple-but-inefficient solution and a more sophisticated solution, and this choice has important structural implications. The architectural design of a large enterprise involves any number of these choices.

Furthermore, a given organization has a finite capacity for managing sophistication and complexity, and a limited willingness to invest in the management costs and overheads that may be required to extract the full benefit from more powerful systems.

There are two agendas that each appear to address opposite sides of this situation. The first is the simplification agenda, advocated most persuasively by my friend Roger Sessions, which addresses the problem that most organizations and their support systems (including IT) are riddled with unnecessary complexity.

The second is the maturity agenda, which suggests that an organization can extract more benefit from certain systems and practices and technologies by becoming more sophisticated. Maturity models are typically arranged as a series of maturity levels, and should be derived (although I suspect many of them aren't) from a robust theory of organizational change and technology adoption.

There are some subtle and usually ignored issues in the interaction between these two agendas, but I don't want to go into these issues here. Instead, I want to point to a couple of critical questions

• how much complexity and power does a given enterprise need (requisite variety)
• how should this complexity and power be distributed and connected

The first question looks like a simple engineering question, which should be bread-and-butter for requirements engineers. The need for complexity and power is driven by some set of demands, together with an economic argument that links cost to value (possibly but not necessarily expressed in financial terms).

The second question is clearly an architectural question. Two things are intuitively obvious about the design of complex machines: not every component needs the same amount of power and complexity; however, some components need proportionality of power and complexity. (It doesn't make sense to put a massive engine in a tiny car, and a fast car needs more powerful brakes than a slow car.)

Power and complexity are higher-order examples of so-called non-functional requirements. Architects need to be able to reason about the composition and decomposition of non-functional requirements.

• Let P be some property of systems and components - say performance or quickness or reliability or security or throughput or whatever. If the components of a system have properties P(1) ... (Pn), what can we say about the composite property P of the whole system?

• Conversely, if we have a requirement for the whole system to have property P, what can we say about the properties P(1) ... (Pn) of each component?

The enterprise is a sociotechnical system, so I'm not just talking about technical components here. A racing car needs a driver whose reaction speeds are matched to the speed of the car, and a maintenance team capable of rapidly diagnosing and fixing all the minor technical problems that could occur during the race.

Similarly, an enterprise needs a set of management capabilities whose intelligence is matched (proportional) to the power and complexity of the operations and operational systems - thus we have an architectural view of the enterprise as an intelligent sociotechnical system, with an appropriate balance of power and complexity.

I don't want to hear that the complexity has been completely removed from the technical systems, because I then fear that the necessary complexity has been merely displaced onto the people inside the organization - or worse, onto the customers.

Conversely, I don't want to hear that the complexity has been completely embedded inside the technical systems, because I then don't trust the people inside the organization to fully manage this complexity.

What I want to hear is that there is just enough complexity and power in the technical systems AND at least enough intelligence in the human systems to manage this complexity and power properly. And I expect the enterprise architect to be responsible for maintaining this proper balance.

Brownfield Requirements Engineering

Some partial and selective notes from the RESG meeting on Brownfield Requirements Engineering, held at BCS headquarters on October 12th 2010. By Richard Veryard.

---

Chairman's Introduction

Ian Alexander opened the meeting by pointing out the gulf between the demands of real engineering projects (95% brownfield) and the body of knowledge about requirements engineering offered by academics and authors (95% greenfield), and sketching some of the characteristic features of brownfield requirements. The importance of this topic is demonstrated by an excellent turnout for this meeting.

Brownfield Systems Engineering

Ian Gallagher of Altran Praxis presented some experience from several large systems projects, both military and civilian. The typical scenario is a major upgrade of existing systems to improve performance and deal with obsolescence; the upgrade may therefore include different types of requirement

• doing new things
• doing old things in new ways
• migrating away from ageing technologies, proprietary systems or restricted technologies (e.g. those subject to export licenses)

The upgrade may be regarded as a single large increment (from AS-IS to TO-BE) but is often managed as a series of smaller increments.

With greenfield engineering, the requirements of the engineering project are pretty much the same as the requirements of the TO-BE system, but brownfield engineering introduces a critical choice: whether to write total requirements for the TO-BE system or to write requirements for the project (or separate increments) based on the difference(s) between AS-IS and TO-BE. IanG expressed a strong preference for the former, but acknowledged that this isn't always acceptable to key stakeholders, especially if the effort would be perceived as excessive. But in any case it's obviously important to be clear what kind of requirements we are talking about.

Beyond the scope of the TO-BE system, there may be a larger system-of-systems context, in which other equally large systems are the subject of equally large brownfield engineering activity. This raises questions of the interoperability of the increments, and the coordination between autonomous brownfield engineering projects, which is another important issue for brownfield requirements engineering.

A key issue is the pressure to reuse and repurpose existing assets. There are three possible reasons for this.

1. Rational economics - genuine cost-saving based on lifetime exploitation of assets
2. Irrational inertia - desire to justify past investment decisions, resistance to change
3. Commercial - vested interests from key stakeholders (such as contractors) to maintain proprietary dependencies

Requirements such as migrating towards more "open" architectures are therefore not only relevant for an immediate brownfield project but also have important implications for the economics of future brownfield projects.

Managing Brownfield Financial Software

Phil Cantor of Smartstream then talked about his past and present experience managing and maintaining software products in the financial sector. (Most of his examples came from previous companies he had worked for.) The requirements of such products don't come exclusively (or even primarily) from the "users" but from a body of knowledge that the package vendor is expected to master. Phil gave the example of an obscure piece of financial calculation that is now only required in the Philippines: users elsewhere in the world may not know about this, but a package that doesn't cater for this requirement will be unacceptable to a global bank.

The package vendor then has to balance a wishlist of enhancement requests and bug reports from hundreds of user organizations, with the practical constraints of maintaining - and hopefully improving the quality of - an existing body of software.

For me, one of the most interesting issues raised by Phil was when he talked about the platform and its requirements. There is an internal platform supporting the user-facing components of the product suite, containing common services and suchlike, and the requirements for this platform cannot be derived purely from the requirements of the package as a whole. Furthermore, the package as a whole serves as a platform for the business processes of the user organizations (Phil's customers), so the same question arises at that level as well. However, this is not particularly a brownfield issue.

Phil also mentioned the challenges of training. From a sociotechnical perspective this is a major brownfield issue, since the performance of the TO-BE system will be affected by the user habits and expectations (e.g. knowing where to find things) carried forward from the AS-IS system. Training is a solution (and often not a very good one) to some set of sociotechnical requirements, and there may be better ways of conveying a new set of habits and expectations to the users and technical staff, but clearly we need to have an understanding of these requirements as well.

Discussion

Lots of further issues came up in the discussion, and I didn't manage to capture half of them. Someone raised the question of scale - what do you do with a brownfield situation with a complex network of interdependent pieces of hardware and software, with the need for upgrades constrained by small budgets and massive complexity. This led to the question of timing - how do you decide whether to upgrade something now or to defer the upgrade until next year. (Nobody actually mentioned the concept of technical debt, but that is clearly relevant here.)


Finally, my memory and notes can be supplemented by a few choice tweets.

@rmonhem much debate over how much to document requirements with significant re-use of existing applications

@rmonhem most difficult challenges have often been posed by cultural, commercial and legal factors.

@jiludvik Bit too much focus on techniques documenting and signing off detailed reqs. This is not specific for brownfield projects!

@jiludvik Phil Cantor's pres has been very entertaining and spot on for brownfield req's eng. I can certainly relate to many challenges he mentioned

Intelligence as Requirement

Let's suppose an organization suffers from a number of the symptoms of organizational stupidity and decides to do something about it. (Not an easy decision, but let's start from there.) How do we determine the detailed requirements for organizational and technical systems change?

Many brownfield requirements analysis projects start with negative requirements - we want to eliminate this bottleneck, reduce this wastage or variation, take out this cost and risk. This involves a combination of factual observation (some bottleneck exists in the target system, with such-and-such consequences) and a value judgement (the bottleneck and its consequences are undesirable relative to some value system).

These negative requirements then need to be converted into positive requirements - we want to improve certain whole-system properties (such as organizational intelligence), and we think we can achieve that by making certain changes to the whole enterprise (regarded as a sociotechnical system). These changes may include engineering new or improved mechanisms, as well as improved use of the existing mechanisms.

Typically, there are many different things we could consider doing to improve these whole-system properties, so we produce a wishlist of potential requirements. But such wishlists create all kinds of difficulties. It may not be feasible to do all of the items on the wishlist, firstly because of limited resources and secondly because some of the items may be mutually incompatible or redundant. So we try to create a meaningful subset of the wishlist, to produce a set of real requirements that is both feasible and affordable.

If the items on the wishlist were all completely independent of one another, then we could categorize each item into subjective categories (say "essential" and "desirable"). Or perhaps we could perform a cost-benefit analysis on each item, calculating a risk-weighted accounting value of the item in terms of its ROI or NPV. This would allow us to rank the wishlist with the most important, valuable or critical requirements at the top; the final set of "requirements" is then defined as that subset of the possible requirements above some arbitrary cut-off line.

Of course both the subjective sorting of requirements ("essential" versus "desirable") and the estimation of costs, benefits and risks are evaluated differently from different stakeholder positions, so the final list is typically fought over between stakeholders. Furthermore, things change over the duration of an engineering project, for various reasons, so the cut-off line comes under pressure.

But a more fundamental problem with this way of defining requirements is that the items on the wishlist are usually mutually dependent, at least to some extent. The costs, benefits and risks of each item may vary according to which other items are chosen, and the number of possible permutations is astronomically large. Therefore the challenge of defining a meaningful and feasible subset of the original wishlist becomes an architectural one - finding some underlying deep structure within the wishlist that permits stakeholders to make intelligent and informed choices.

Can requirements be "solved"?

Much writing on systems architecture, requirements engineering and related disciplines starts from the premise of a distinction between problem-space and solution-space. This is an extremely useful distinction under certain conditions, but we are increasingly facing challenges in which these conditions no longer hold valid. Here are some examples.

1. Where the goal is not to design some complex artefact but to deliver some set of outcomes. For example, to make an organization more efficient or intelligent, to make a tax system more equitable, or to make a community more self-sufficient.

2. Where the target system is complex and governed by conflicting value systems, so that the challenge is to formulate and negotiate a series of interventions, from which a useful and acceptable change to the target system might emerge.

3. Where the agency for change is embedded within the target system, rather than being external. Anyone who gets involved in the project becomes part of the target system, and it no longer makes sense to ask the popular question "are you part of the problem or part of the solution?"

---

One of the attractions of the problem/solution distinction is that it helps to build agreement around a statement of the problem, and focuses the debate on different ideas and preferences for solving the problem. But to see the limitations of this distinction, we only have to look at the political sphere. There is surprisingly little difference between the political parties in their statement of the fundamental problems facing the government; but there are wide differences in the policy instruments they advocate to solve these problems. Of course, these differences are not just alternative sociotechnical designs, but are produced by different belief systems (how the economy really works, what are the root causes of crime and terrorism, and so on) as well as different value systems (what kinds of prosperity and well-being matter most).

So is there a discontinuity between highly complex and messy sociopolitical problems on the one hand and relatively simple and self-contained engineering problems on the other? To answer this kind of question, some people appeal to a simplistic interpretation of the Cynefin framework, treating it as if it were a simple 2x2 matrix mandating different problem-solving or problem-tackling methodologies according to the degree of complexity in the problem domain - but of course this kind of answer begs lots of questions about the relationship between problem and solution which a more subtle and sophisticated application of Cynefin should expose.

Any human system (including organizations) will display some degree of complexity and messiness, and it is in this context that the concept of "requirements" needs to be understood. In a separate post, I shall explore how such a concept of requirements applies to organizational intelligence.

---

Requirements engineering is about the interaction between five kinds of system. In the problem-solving paradigm, these fit together as follows.

• A deliverable system is designed to be a "solution" to some "problem". (In a simple case, this could be a single complex artefact or a coordinated series of interventions.)
• A target system is where the "problem" is located, and where the "solution" must fit. (In a simple case, this could be an organization unit or operational business process.)
• A value system provides the criteria justifying why the problem is a problem, and why the solution would count as an improvement. (In a simple case, there is a single sponsor whose value system is paramount.)
• A belief system, an explanatory model or theory that explains the cause of the problem and the plausibility of the solution. (In a simple case, everyone broadly shares the same belief system.)
• An intervention system, a force of people and resources mobilized to do something relevant. (In a simple world, the intervention system is a project team under the exclusive governance of a project sponsor with a unified and dominant value system.)

Within the problem-solving paradigm, the relationships between these five are pretty straightforward, and it may not be worth articulating and analysing them separately. However, as we move beyond the problem-solving paradigm, these become plural rather than singular, and the relationships between them become far more interesting.

---

See also

• Niklas Björnerstedt, The obsolete idea of requirements (Feb 2010)
• Richard Veryard, Zero-Based Requirements (Sept 2010)
• Richard Veryard, Third-Order Requirements Engineering (Slideshare presentation based on a paper I wrote in 1995 with John Dobson).

Making Conversations Visible

@sbskmi gave an excellent presentation at the RESG AGM yesterday evening, offering a survey of sensemaking tools.

Simon's research group at the Open University has a tool called Compendium, which belongs to a long and respectable line of issue and argument mapping tools, going back to IBIS (Horst Rittel, 1972). Simon showed a range of recent initiatives that demonstrate that issue and argument mapping has, as Simon puts it, "come of age".

Compendium itself has been designed to allow logical arguments to be supported by rich media evidence, such as video. So we can manipulate conflicting knowledge claims, together with the ethnographic material that might be relevant to resolving these claims.

The particular interest of tools like Compendium (and its Web 2.0 cousin Cohere) to the Requirements Engineering community is the desire to document design rationale, and these tools can certainly be used for this purpose. But they can equally be used to support real-time control systems, and Simon showed an emergency response coordination scenario, with web service links to various information feeds, as well as the ability to produce mashups of various kinds.

However the aspect of Simon's work that interested me most was the potential link to organizational intelligence. For example, he displayed a model of creative competences for complex challenges, based on the work of Palus and Horth, and showed how Compendium could use visual images to support the Palus and Horth methodology. (There are some parallels with the Repertory Grid technique, which I introduced into data modelling in the 1980s.) Simon also showed how Compendium could be used to compose machine intelligence with human intelligence. This helps to realise the original vision of Takehito Matsuda, who twenty years ago defined organizational intelligence as "the interactive-aggregative complex of human intelligence and artificial intelligence in an organization".

As Brenda Dervin argues, sense-making is triggered by anomalies and exceptions. The point about an exception is that it forces us to review and revise our pet models and theories, or at least it should do so. As Lakatos pointed out however, in his brilliant essay Proofs and Refutations, people typically deploy various tactics for dismissing exceptions in order to preserve their favourite models and theories, including Monster-Barring and Monster-Adjustment. (See my piece on Models and Monsters.) Thus the relationship between any given piece of evidence and a complex argument is an act of interpretation and is itself subject to argument.

To understand an argument or rationale, we need to pay attention not just to the domain (subject matter) of the argument but also the discourse or discursive practice. Within a large organization, there are several competing discourses, and the intelligence of the whole organization depends critically on a healthy interchange between different discourses. For example, arguments based on conventional accounting practice may bias the organization towards certain ways of solving problems, and make some kinds of innovation impossible; so sometimes management needs to be able to step away from the accounting paradigm and look at other kinds of rationale for organizational change. It would be interesting to see if models and software tools could support a conversation that straddled multiple discourses.

But in any case, these tools for collective sense-making and decision-making should fit nicely into an overall architecture for organizational intelligence.

---

Making Conversations Visible

View more presentations from Simon Buckingham Shum.

Zero-Based Requirements

Brown-Field Requirements

One view of requirements engineering is that its purpose is to produce a complete and coherent statement of what some system-of-systems is required to do - in other words, its behaviour in the broadest sense, including "functional", "non-functional" and "commercial" requirements.

In a "Green-Field" scenario, we might imagine that this statement of requirements would result in the procurement and installation of a system of systems meeting the stated requirement. Requirements engineering is focused on understanding exactly what is required, and specifying it in an unambiguous and testable form.

But in almost all engineering projects, some system of systems - technical or sociotechnical - already exists, and the practical purpose is to make some planned changes to it. So people in the RE community are starting to talk more about "Brown-Field" requirements.

(RESG Event: Managing Brownfield Project Requirements, London, October 12th 2010)

Gap Analysis

An obvious starting point for brownfield requirements analysis would seem to be the identification of a gap between desire and reality. People often produce two models - an AS-IS model that describes the existing system, and a TO-BE model that describes its future replacement. The engineering requirements are then derived from the differences between the AS-IS model and the TO-BE model. This will typically result in a solution, possibly involving rebuilding some of the subsystems, replacing or upgrading some of the component parts, adding some new stuff or stripping out old stuff, rewiring the network, retraining the people, resetting system policies and parameters, and so on. In addition to a solution blueprint, showing how all these elements are to be configured, there will also be a transition strategy, indicating how (and in what sequence) all these changes will be installed. There are usually operational constraints - for example, a requirement to keep critical business processes running at an acceptable level during the transition period.

Imagine you want to rebuild your kitchen. You have to think about fitting new units into the existing space, or possibly moving a wall to give yourself more space. You have to decide whether you are going to keep the existing fridge (which you only bought last year) or buy a new one. And you have to think about how long you can manage without being able to cook. Moving the wall, or deciding to keep the old fridge, belong to the solution domain. But if you are going to do requirements analysis properly, there needs to be something in the statement of requirements that helps you determine these aspects of the solution.

In all but the smallest and most simple projects, there will be many solution variants. The decision to retain or replace a particular component may be based on a technical calculation of its likely performance and capacity within the new configuration, or may be based on a political calculation as to the most convenient budget from which to fund the replacement (in other words, preferably someone else's budget, if we can get away with not replacing it now).

Ten years ago this month, I wrote a piece about this in relation to Component-Based Software Engineering. Supply and Fit (CBDI Journal, September 2000).

But there is a more fundamental reason why there are many possible solutions - because making sustainable changes to complex systems is a tough challenge. Large and complicated change programmes aren't always the most effective; a small intelligent fix is often far better (and less risky) than any amount of optimistic meddle.

So before we can get to a solution blueprint and a transition strategy, we need an intervention strategy. This takes us out of the comfort zone of requirements engineering into general systems thinking.

Leverage Points

Donella Meadows identified twelve leverage points for making changes in complex systems, and suggested that these could be ranked according to their power. See original paper by Donella Meadows. A version is included in her posthumously published book Thinking in Systems (2008).


If the intervention strategy can be expressed as a combination of leverage points, then this raises the question for requirements engineering - how do we work through the requirements of changing a complex adaptive system in a way that could produce this kind of intervention strategy?

Zero Based Procurement

Finally, I wanted to make a comment about one of the (many) dysfunctional aspects of prevailing procurement practices. In his blogpost Was this NHS IT tender a stitch-up? (Computer World, September 2010), Tony Collins talks about the difficulties of referencing a specific product or system in a tender document. "If a user organisation has a system it’s happy with, and wants to keep and enhance, why would it want to go through the needless expense of an EC tendering, rather than simply renew the contract?"

Procurement rules may have been designed to prevent cosy and uncompetitive relationships between public sector organizations and their suppliers. They appear to have the effect of forcing each procurement to be treated as a separate exercise, starting each time from a blank sheet of paper, so that there is at least the theoretical possibility of giving new suppliers a chance. (This is similar to the principle of Zero-Based Budgeting.) Many people doubt that these mechanisms actually have any real effect on competition or value-for-money; but meanwhile, these mechanisms appear to have a strongly negative impact on through-life capability management. How can brownfield requirements engineering be done properly under these constraints?

Hard Cases Make Bad Law

David McCoy complains about More Loopholes in legislation. Apparently the State of Nebraska is making an unplanned collection of abandoned teenagers, because it forgot to put an age limit on its abandoned child legislation. [CNN 14 November 2008]

My friends in the Requirements Engineering community look at the legislative process with despair. A single piece of legislation is a highly complex set of provisions intended to produce a given set of outcomes. It is not just ambiguity (David McCoy's particular bugbear) that causes problems, sometimes the problems are caused by clumsy and complicated attempts to avoid ambiguity. Feature interaction between these complicated provisions within a single piece of legislation, let alone multiple separate acts, often creates additional loopholes. New legislation is constantly required to correct the unintended effects of previous legislation. Systems engineering methods (or perhaps systems-of-systems engineering) would surely make a big difference. Despair or opportunity?

(However some stakeholders, such as those lawyers and accountants whose livelihood depends on selling knowledge of these loopholes to their clients, seem to have little incentive to improve matters. Legislative bodies are packed with lawyers and accountants. Funny thing that.)

But there are some other important lessons for complex systems of systems here. All states have some legislative provision for abandoned children, but Nebraska is the only state that didn't specify an age limit. Hence children who are too old to be abandoned in other states are being driven to Nebraska and dumped there. This is similar to the situation of a service provider who tries to offer an unlimited service when most other service providers only provide a limited service, and ends up with a skewed distribution of demand. Think broadband - the service providers who offer unlimited broadband are going to get a lot of high usage customers. (So the economic viability of a given service, at a given price and service level, may depend on the actions of your competitors.)

Nebraska is therefore under peer pressure to fall in line with other states. In a connected world there isn't much scope for local autonomy, if it means that a state offering a better service gets dumped on by its neighbours. Nebraska is therefore likely to introduce a 3-day or 30-day age limit.

So what happens to the problem children who are older than 30 days? It's not as if their problems can be eliminated by a simple stroke of legislation. The families who have been dumping their children in Nebraska clearly have serious problems, and perhaps Nebraska has decided it doesn't want to help them. So which system takes over responsibility for these cases?

And don't say there might not be a system at all. I reckon some kind of system is inevitable, but it just might not be a very nice system. The underworld has plenty of systems: of crime and drugs and prostitution and abuse. In a complex distributed world, there are often stakeholders whose goals and values conflict with ours, and this is all grist for the requirements engineering mill.

David suggests that legislators should write laws assuming the worst. I agree that there needs to be a much more rigorous analytical process before designing any complex system, and this needs to start from the desired outcomes. It also needs to include some kind of environmental analysis, understanding the potential interactions between a given system and other systems, including potentially competing or hostile ones. But I go further: each piece of legislation needs a well-defined scope, and this should ideally come from some kind of overall political architecture (equivalent to an Enterprise Architecture).

Goal-Oriented Requirements Engineering and its relevance to SOA

The i* approach

Last week I attended a seminar organized by the BCS RESG. Modelling your System Goals: The i* approach. The main speaker was Eric Yu (University of Toronto), the original developer of i*. The remaining speakers reported their experiences using and extending i* in a range of projects.

The i* approach is a methodology for goal-oriented requirements engineering (GORE). Professor Yu characterized the distinct nature of Goal-Oriented RE as follows.

Conventional RE	Goal Oriented RE
Describes how things work (AS-IS) and how they should work (TO-BE).	Describes why things work (or should work) that way.
Models of behaviour.	Models of intention and rationale.

i* is particularly focused on so-called "early stage" requirements engineering - before you know what "the system" is going to be. It involves two key models: strategic dependency models and strategic rationale models.

The strategic dependency model identifies goal dependencies between actors. For example, a CarOwner actor depends on "CarBeRepaired", and this can be fulfilled by the BodyShop actor. Put crudely, a goal dependency consists of a pair: {"I want", "I can"}.

i* distinguishes between two different kinds of goal dependencies: hard goals and soft goals. Although this distinction was explained in terms of precision, this seems unsatisfactory to me, since surely precision is itself a matter of degree and timing and negotiation. However, the soft goals that were used as examples seemed to me to possess some other interesting features, which may be relevant to making this distinction clearer. Firstly, the soft goals were generally not amenable to precise specification by the consumer of the goal. Secondly, there is a sense that what counts as satisfaction of the goal is context-dependent and shifts over time. Thirdly, the perceived outcome would generally affect the relationship of trust between the consumer and the provider, and thus the future behaviour of the consumer towards the provider.

This led me to an alternative way of formulating the distinction - according to who owns the goal. In some cases, the consumer determines and decomposes the problem, and then delegates the solution. I think this produces a hard goal that is (or should be) relatively easy to specify. In other cases, the consumer delegates the problem (formulation of) as well as the solution (provision of). I think this would produce what i* calls a soft goal.

So for a soft goal such as FairSettlement, what counts as fairness is delegated either to the service provider, or to a trusted third party such as a regulator. (In some cases, notions of fairness are implicitly delegated to virtual third parties, such as TheCommunity or PublicOpinion, but these are notoriously unstable and unreliable mechanisms.) The significance of this distinction for goal modelling is that identifying something as a soft goal introduces an additional set of concerns for the requirements analyst.

The strategic rationale model identifies causal connections between goals and subgoals for a given actor. Note that the behaviour of an actor may depend on perceived causal connections and associations, rather than actual ones. So if we are going to model rationale properly, we need to include some representation of the actor's beliefs. However, last week's i* presentation did not introduce any notation for belief.

SOA interpretation

Much of the material presented was pre-SOA in outlook, and there were some limiting assumptions about the nature of the systems being built. However, I was looking to push i* beyond these assumptions. What I wanted to explore was the possibility of using i* to design services and service-oriented solutions.

There appears to be a very straightforward mapping from goal dependencies on the strategic dependency model to business services. And these requirements are supported by an understanding of the rationale of each actor. The reason for this is that we want to design business services in terms of added value, and this seems to rely on our having some notion of value from the perspective of the service consumer. SOA is geared towards flexibility, and an appreciation of the possible rationale of each actor also helps build solutions that support a good range of different scenarios.

Another SOA-related motive for modelling dependency and rationale is to analyse compliance (including so-called non-functional requirements). For example, we may identify some system vulnerability, and then identify a reciprocal dependency that provides some enforcement mechanism. We can then look at the system dynamics within a collaboration, to determine the adequacy of this mechanism, as well as any possible side-effects.

In general terms, I am convinced that some modelling of intentions and outcomes is an important aspect of modelling for SOA. In the Boxer-Cohen Triple-Articulation approach to Asymmetric Design, this is known as the Deontic Articulation. i* is much simpler, and widely practised - but is it powerful enough?

Complexities, difficulties and future opportunities

Modelling intentions. There are difficulties involved in modelling the intentions of both organizations and machines, and these difficulties cause some people to say that it doesn't make sense to attribute intentions to either organizations or machines - only to people. I don't agree, for three reasons. The first is that many of these difficulties also apply when we are trying to understand the intentions of people, since people often have confused or concealed intentions. The second is that there are established techniques for tackling these difficulties, which help us to understand the intentions of all kinds of system, at different levels. The third is that we simply cannot make sense of business strategy and organization at a reasonable level of abstraction without somehow talking about organization goals.

Goal mismatch. In practice, there is unlikely to be a neat and exact match between "I want" and "I can". There may be both pragmatic mismatch (crudely: what the provider does is not exactly what the consumer wants) and semantic mismatch (crudely: what the consumer understands is not what the provider understands). This is a form of impedance or asymmetry.

For example, a car driver's real goal may be to have a reliable car with low total maintenance cost. But no service provider offers exactly this service. So the car driver has to compose something that approximates to his real goal, using a combination of car maintenance services and car warranty (insurance) services. (There may be significant commercial opportunities for a value-adding service or service platform in this kind of space. Hence the relevance of goal modelling to business strategy.)

Within i*, the strategic dependency assumes an exact match between the consumer's view of the goal and the provider's view of the goal. To the extent that there is some unfulfilled remnant, this is analysed within the strategic rationale of that actor. But it is not clear to me how this approach would support architectural reasoning about the unfulfilled remnants and the strategic opportunities for developing new added-value services.

Deconfliction. Given that a complex situation contains considerable goal conflict, we need a systematic way of resolving goal conflict. Deconfliction means organizing operations in a way that minimizes the potential risk of interference and internal conflict. Conversely, we need ways of resolving goal synergy.

Note that from a security perspective, goal synergy between (supposedly) independent actors may not be a good thing - especially where it indicates opportunities for collusion and fraud.

Variation. A robust service economy typically needs to accommodate considerable variation in the goals and rationale of the actors. So instead of modelling a single standard strategic rationale, we need to understand the nature of the variation between different rationales, and the implications of this for producing robust and agile systems.

For example, a healthcare system makes some assumptions about the rationale of a patient. But a patient that happens to be a professional athlete (with particular concerns about stamina and performance) will have a very different strategic rationale in relation to healthcare, as compared to a patient that happens to be a healthcare professional (with above average exposure to infection and other risk).

Dynamic and recursive rationale modelling. The causal relationships between goals and subgoals are subject to dynamic effects. For example, many processes operate in terms of surrogate goals - outcomes that are valued not for their own sake but because they are correlated with some real goal. The surrogate goals are available for measurement before the real goals, and may therefore provide useful predictive metrics. (See discussion of Surrogate EndPoints in relation to SOA Pharma.) This means that cause-effect modelling may need to include system dynamics such as delay and oscillation.

There is also a problem that the rationale of one actor may depend on that actor's beliefs about the rationale of another actor. For example, Professor Yu presented an example from insurance, where reorganizing the strategic dependency model could align the insurance broker with the interests of the customer (instead of the broker being aligned with the interests of the insurance provider). In this situation, what the insurance customer demands from the insurance broker depends crucially on the customer's belief as to whose side the broker is on. But that in turn may depend on the insurance broker's beliefs about the customer's acting in "good faith", and about the future commercial tactics of the insurance company. So the whole thing becomes recursive, rather like the Knots identified by R.D. Laing.

Summary

I think the overall approach of i* is extremely interesting for service-oriented architecture, and is broadly consistent with the general approach to business modelling for SOA we have been developing.

---

A version of this post was published in the RESG Newsletter Requirements Quarterly (June 2005).