Business Analysis Skills

Andrew Johnston comes away from the 2008 Enterprise Architecture Conference in London worried about A Shortage of Analysts.

One of the major causes of project failure is that the requirements aren't properly understood and communicated, but this isn't exactly a new problem. What is new is that emerging development paradigms, from SOA to outsourcing, affect the opportunities for requirements analysts to learn their craft. In the past, analysts and developers typically worked together in the same organization - developers consumed and actively critiqued the work of the analysts. (When I was a programmer, I asked so many awkward questions of the analysts they decided to make me an analyst instead.)

However, although the experience of collaboration between analysts and developers is undoubtedly useful in the acquisition and refinement of analysis skills, and it is important that analysts have an appreciation of "making it work", I am not sure I agree with Andrew that analysts need to have a development or equivalent background. Someone might equally argue that analysts should have a systems testing background - "making it not work" - because that would give them a strong sense of what can possibly go wrong with an apparently simple idea for business improvement.

Twenty or thirty years ago, it may have made sense to recruit business analysts from the ranks of developers. After all, they were the only people in the organization who understood IT, they were often better educated than the "end-users" (in those days we were mostly dealing with automating administrative systems, so the end-users were mostly clerks), they could think logically, and they had a kind of intellectual curiosity that had lured them into IT in the first place.

But these assumptions no longer hold. Intellectual ability and IT literacy are certainly not unique to the IT department - in some organizations nowadays the IT department comes a long way behind the "user" departments in these qualities. And it is in the user departments that managers have experience of making a business process work - surely this is more relevant to business requirements analysis than making a computer system work?

The General and The Particular

In his post What do you need to know to be an IT Architect? Simon Guest (editor of the Microsoft Architecture Journal) identifies some key elements of architectural knowledge: IT Environment, Business / Technology Strategy, Design Skills, Quality Attribute Skills, and Human Dynamics.

I think there are two important clarifications we can make here.

Firstly, Simon's starting point is, rightly, know-how (skills) rather than body-of-knowledge (theory). Nick Malik makes a similar point in his post on Enterprise Architecture Interview Questions: "proven abilities and past experiences, and not book learning".

The distinguishing characteristic of an architect, in my view, is a certain way of paying attention to a certain class of problems and opportunities. (Just as a chess grandmaster has a vast body of knowledge of past games and variations, as well as a complex mental library of patterns, but these are only useful for playing the game of chess.)

Therefore the architect's knowledge (of the organization, of the business) needs to be active knowledge, what Vickers called appreciation.

Secondly, when Simon talks about "the organization", "the business", and so on, these terms can be interpreted in two ways. Does the architect need to know about organizations-in-general, business-in-general? Or does the architect need to know about this particular organization, this particular business?

Ali Arsanjani has characterized this choice as one between two philosophers: Kant versus Hume. Do architects concentrate on some apriori (generalized, enterprise-wide or industry-wide) schema (Kant), or do they remain grounded in the specific local requirements (Hume)?

(Some people refer to this choice as top-down/bottom-up. But I think this terminology is vulnerable to wide misunderstanding, because different people have different orientations - is the business at the top, or is the generic model at the top? See my post What Does Top-Down Mean?)

I believe the best architects are those that can do both - who have both general knowledge/know-how and particular knowledge/know-how, and can make intelligent connections between the general and the particular. In what ways is this bank like any other bank, and what particular things differentiate this bank from other banks?

That's how we train our SOA architects to think.

Service Design

Who should design the services in a service-based business? And what kind of capability is required?

Clearly many people in the IT industry focus on software services (SAAS), and see service design as an IT capability.

Meanwhile there are people in the design industry, who see service design (and interaction design) as exciting new areas for applying their design skills.

See the Design Council (UK) and the Service Design Network (USA). See also blogs by Peter Merholz and Dan Saffer.

It would be great to see more collaboration between these two communities.

Technorati Tags: design SaaS service service-oriented

Carrot and Stick

Motivating Reuse

From a software perspective, one of the possible benefits of SOA is reuse.

The notion of "reuse" is inherited from previous waves of software technology: module, object, component. Over the years, we have been told many times of the substantial economic benefits to be gained from reuse (greater development productivity, substantially reduced cost, faster delivery, and so on).

And when the actual scale of reuse turns out much lower than the apparent potential, we are given a range of explanations (excuses). Perhaps that technology wasn't quite powerful enough, but this new technology will release floods of reuse. Perhaps the technology was fine, but the people weren't properly trained or managed. And perhaps this additional technology will force the people to deliver reuse.

If we want people to do things differently, we have to consider the three Es.

Enabling	Do the people have what they need to do things differently? Skills, resources, tools, infrastructure, support?	For example, if the technology supporting a different way of working is clumsy and immature, it may be impossible to adopt this way of working without compromising speed or quality.
Encouraging	Do the people (and their managers and supervisors) have a reason to do things differently? Motivation, reward (or sanction),	For example, if a project manager's status depends on having a large project team, this conflicts with any incentive to improve productivity.
Empowering	Do the people have the power to do things differently?	For example, if people are forced to work on narrow, short-term projects, under strong design constraints, they may not have the opportunity to produce reusable artefacts.

According to Jon Udell, Verizon paid particular attention to questions of motivation in implementing its IT Workbench.

Stick	Project funding is contingent on the contribution of sharable services to the repository.
Carrot	Developers required to use the repository -- like all employees required to use centralized corporate services -- are effectively paying a tax. And like all taxpayers, they want to see their tax dollars put to good use. Verizon says that its IT Workbench rewards developers by wrapping useful capabilities -- auditing, debug tracing, security -- around the services they develop.

source: SOA heroes

Udell also refers to the public recognition and praise accorded to those developers who adopt the desired practices, and quotes Blue Titan's Frank Martinez "Make them heroes". However, I have a slight qualm about this, to the extent that reliance on heroes is a sign of an immature software practice.

Motivating Sharing

But I no longer think that reuse is quite the right concept for SOA. It raises all sorts of technical issues about deployment and instantiation of a service, which are not directly related to the grand economic benefits of software reuse. People are starting to talk more about sharing services rather than reusing software.

Many of the enabling / encouraging / empowering questions are still valid. Of course we must look at the transaction costs of sharing, including both the costs to the individual (effort, learning, moving outside comfort zone) and the costs to the organization (trust, complexity). Sharing potentially exposes us to additional interoperability risks, which need to be properly understood and managed.

The main advantage of sharing over reuse is that it is much more meaningful to the business. Software reuse is fine as an abstract idea; but sharing services has much more concrete implications for business users, whose support can be mobilized to help encourage and empower SOA developers.

Technorati Tags: interoperability reuse SOA service-oriented

Web Service Management

According to a recent commentary by my CBDI colleagues, web service management is stuck in the chasm (21st October 2004). In this piece, they discuss web service management from a tool perspective - what are the vendors up to. But we also need to think about the web service management process. Who is responsible for web service management within an SOA environment, what is it they do, what skills and techniques do they need?

When I spoke to IBM recently, the answer to the first question was unequivocal. An SOA environment needs a service architect. The service architect is the primary user for web service management tools, such as the service management dashboard.

For many organizations, service architecture is a fairly new role, and the architectural responsibilities of web service management are not widely understood. Although an increasing number of individuals and teams are becoming adept in the detailed mechanics of web services, large-scale applications of SOA principles remain comparatively rare, and so not everyone has yet had the opportunity to get first-hand experience in addressing large-scale architectural issues.

The service architect plays a key mediating role (“broker”) between development and operations, and is responsible for the negotiation of a manageable set of services with defined service profiles.

A service profile is a recognised pattern of service interactions (an orchestration scenario). A service profile distribution is a statistical pattern indicating the occurrence of service profiles. (For example, this might allow the system to generate an alert if a given exception condition is executed for more than x% of cases. More generally, it supports Statistical Process Control.) Deviation from accepted service profiles or profile distributions may generate an alert, demanding someone's attention. Alternatively, the alert may be passed as a signal to the web service platform, where it triggers some autonomic response.

If you only have a handful of web services, then the architecture role seems almost unnecessary. The developers can define the profiles, and pass them directly to the operator for monitoring and management. But if you have a large quantity of web services, composed in multiple and complex ways, then managing the complex interrelationships between services at different levels is beyond the scope of either the developer or the operator, and a separate service architecture role becomes inescapable.

A key job for the service architect is to create a service geometry - to organize the services into a composition hierarchy, identifying and stratifying meaningful and manageable clusters of composite services. Profiles can be defined and monitored at each level of the service composition hierarchy, with the architect responsible (sometimes) for dividing up the work of other people into coherent chunks, and responsible (always) for understanding how the chunks interoperate.

But this isn't something you can do on a white board. It calls for geometrical reasoning, of a kind that many IT architects will not have faced before.

The kind of problem faced by the service architect today is similar to the kind of problem faced ten years ago by the network architect. While the network architect was dealing with the topologies and performance characteristics of physical hardware and communication links, the service architect must deal with the topologies and performance characteristics of abstract services and B2B links. There are some techniques and experiences that can be transferred across from one domain to the other, but the job of the service architect is a new one, calling for new ways of coping with the challenge of complexity.

Our next task is to spell out the service management process, and to provide guidance to the service architect. Watch this space.