Showing posts with label BPEL. Show all posts
Showing posts with label BPEL. Show all posts

Saturday, March 18, 2006

Event-Driven

Contained in Edwin K's post BPEL for Applications is a statement about the importance of events.
Business events are important because they represent the glue between business activities and the foundation for realtime analytics. Up until now business events have been second class citizens and often model as a service invocation to some kind of broker. Our goal is to make business events a first class citizen by defining a new event definition language and add the capability to BPEL processes to raise and subscribe to events.
Event is a first-class construct in the business metamodel we have produced for SOA. Notice the feedback loop from Business Intelligence back to Event. I welcome Oracle's declaration of support for this construct, and I look forward to seeing some practical outcomes.

Technorati Tags:

Thursday, March 16, 2006

Grey Matter

One way to get greater economics of scale (and scope) in SOA is to simplify and generalize the basic services by shifting some of the complexity into the wiring between the services.

So I was interested to hear a report on the radio today about recent studies of the human brain. As I understand it, the brain is divided into grey matter (which does the basic thinking) and white matter (which wires everything together). So perhaps we can very crudely equate the grey matter with services, and the white matter with the orchestration of the services.

Now here's the interesting bit - the ratio of white matter to grey matter in different people. Pathological liars have significantly more white matter than other people. (It seems that skilled falsehood requires greater orchestration. It is not clear whether people lie because their brains enable them to lie, or whether some brains develop in particular ways because their owners are exercising the capacity to lie.)

Meanwhile, autistic people tend to have a much lower amount of white matter than other people. There are two characteristics of autism that may be relevant here: difficulty in interoperability, and difficulty with anything other than the literal truth.

Has all this got anything to do with SOA and BPEL? You decide.

Source: BBC Radio 4, Leading Edge, March 16th, 2006
(full programme available online for a week after broadcast).

Technorati Tags:

Tuesday, March 07, 2006

BPM and SOA

At the BCS Business Information Systems SIG last night to hear a talk by Howard Smith of CSC. Meanwhile, I've been listening to an ACM Queue podcast with Mike Vizard and Edwin Khodabakchian (formerly Collaxa, now Oracle). 

I have spoken to Edwin and his (present and former) colleagues in the past. I hadn't met Howard before, but I was already familiar with his work and I am in contact with some of his associates in the BPMI world. 

The following post is something I've been thinking about for a while. I am happy to acknowledge the influence of Howard and Edwin, who both know a lot more about the BPM side than I do. But there are some things I'm saying here that I haven't heard either of them say, so blame me (not them) if you don't understand or agree.


What is the relationship between Business Process Management and SOA?

A good place to start is with the idea that SOA gives you the decomposition of functionality into (standardized) services, while BPM gives you the assembly of these services into (flexible) business solutions.
  1. We can decouple the specification of the process from the specification of the units of work making up the process. (In my Business Modelling for SOA material, I describe this as separating WHAT from HOW.)
  2. We can put the specification of the process into a standard process language. (The preferred choice here is BPEL, for various reasons, although there are some known issues.)
  3. We can automate and distribute the assembly of the process. (With appropriate tools, the assembly of the process or the selection of an appropriate process variant can be done either in real-time or dynamically at the point of need.)
  4. We can then take power to the edge - delegating process management to the people working at the edge of the organization.
  5. Ultimately, not just the process but the process management becomes event-driven. This gives us requisite variety at the process level.
  6. Process architects now need to work at a higher level of abstraction. Instead of specifying a standard one-size-fits-all process, they need to produce what we might call a metaprocess - frameworks and patterns that support process management. They need to pay attention to architectural process issues, such as coordination and interoperability risk.
These ideas are probably some way from mainstream adoption; but there seem to be enough early adopters experimenting with some of these ideas to keep the BPM market moving forward.

Business Process Innovation

The second part of Howard's talk was about Business Process Innovation. If we imagine that BPM plus SOA gives us a reasonable way of automating the business process, then the next challenge is that of automating business process change. 

Howard's working hypothesis is that business process change is driven by problems. CSC has adopted a Russian problem-solving methodology called TRIZ, and has been working on a process-oriented version of TRIZ called P-TRIZ. 

Most of the audience (including myself) were not familiar with TRIZ, and there was a lively discussion about its strengths and weaknesses. My first impression is that TRIZ would not be suitable for all types of business problem, as it seems to lack adequate constructs for modelling complex dynamic systems. However, it does have some interesting characteristics that may make it particularly amenable to automation. Firstly, there is a systematic approach to problem decomposition. Secondly, there is a complete enumeration of solution strategies, and a useful collection of abstract solution patterns. 

Howard showed us a stand-alone prototype tool that supported a simple version of TRIZ. What would be really interesting (and this is presumably CSC's plan) would be to have this functionality integrated into the BPM tools. Then we would be able to have local process-oriented problem-solving, with process architects able to concentrate on the emergent properties of the whole. 

This looks like a significant contribution to the overall BPM/SOA vision outlined above.

Wednesday, November 09, 2005

Twin-Track Governance

Twin-track development involves a management separation between the development of components and services, and the development of larger artefacts that use these components and services. It was a characteristic feature of the more advanced CBSE methods, such as Select Perspective, and is obviously relevant to service engineering as well.

Clarification Update: Select Perspective has always been more than just a CBSE method, and now qualifies as a full-fledged SOA method. Even as a CBSE method, it was one of the first to embrace service as a first class construct. Although this may not be immediately obvious from the Select website, it is clear from an article contributed by Select consultants (then part of Aonix) to the CBDI Journal in 2002.

The twin-track pattern can be superimposed on a traditional lifecycle process, such as that recently propagated by OASIS for the web service lifecycle (pdf). Even though the OASIS process is presented as applying to a single web service, it doesn't take much reframing to see this kind of process applying to an artefact that is larger than a single service - such as a whole system or subsystem - provided it is specified and designed in one place at one time. So we can start to see the OASIS process (or a suitably generalized version of it) applying to either of the twin tracks - but not at the same time. Both tracks may be following some version of the OASIS process, but they don't talk to each other.

The twin-track pattern is sometimes interpreted in a simple way, with an IT organization divided into a service-creation track and a business project track. The service track produces services with web service interfaces; the business projects produce applications with user interfaces (UI). In this interpretation, use of BPEL belongs exclusively in the service track, because it doesn't produce applications with UI.

However, we can interpret twin-track in a much more powerful way than this, by generalizing the pattern. We simply specify that supply of services is in one track, and the consumption of these services (even if this is in the context of using BPEL to build larger artefacts that are themselves services) is in another track. The point of the twin-track pattern is that the supply and consumption can be decoupled and managed separately - possibly even in separate organizations. Of course, this pattern can be applied more than once, yielding more than two tracks in total.

Meanwhile, the possession of a UI is probably of secondary importance here. With SOA, we can (and probably should) build applications that give the user a choice between interacting directly or via some user-built secondary application. Thus for example, I want my online bank to offer me a set of services rendered in two alternative ways: firstly via a UI (probably browser-based) and secondly as a series of webservices (or equivalent) that I can invoke from within some desktop money management program. Think of Google and eBay delivering the same services via browser and via web services. In the service economy, I want all interfaces to have something like a webservice or REST or RSS/Atom alternative.

And perhaps if lots of people are using desktop money management programs, it might be cheaper for the bank to give all remaining customers a low-functionality desktop program (or recommend a suitable bit of freeware) and then decommission the UI altogether. I'm not saying this is always going to be advisable, but it's certainly an option. So we might see a growing number of serious business applications with no traditional UI at all.

In architecture, if you build windows everywhere, it makes it harder to join buildings together. Think of a university campus, which grows piecemeal over many decades. If every new block has a blank wall, then it is easier to build another block next to it. If you put windows on every available wall, you have to put useless space between the blocks, and then build silly walkways to save people keep going down to the ground floor. The evolution of complex SOA raises some of the same issues.

Even with single-track development, there is a governance question here. How can we maintain order over a complex and evolving system, if we cannot simply outline all the requirements at the beginning? And with twin-track, a critical function of SOA governance is governing the relationship between the tracks, however many tracks there may be.

How do we get (and keep) all these loosely coupled development processes and operations processes in alignment with each other, and also in alignment with the business. And alignment with IT accounting (financial and otherwise) would be nice too. Obviously the whole point of twin-track is that you can decouple the tracks to some extent, but if you decouple them too much then you throw baby (reuse, interoperability, flexibility) out with the bathwater (agile=fragile).

Some sources advocate frequent synchronization between development teams. While synchronization does not necessarily rule out federated, distributed development, but this would only be possible with a great deal of horizontal coordination. And this would introduce lots of other challenges.

SOA governance governs what kind of development is appropriate, and how it should be coordinated. For example, SOA governance provides ground-rules for participants to agree interfaces before they go off and do their own thing, and for enforcing these agreements. In the real world, we know that all agreements are subject to being reneged and renegotiated as requirements and conditions change. But who incurs this risk, and who shall bear the cost of any change? If you decide you need to change the interface, am I forced to respond to this, and if so how quickly?

Change management (e.g. avoiding uncontrolled demand for service specification change) cannot be managed within either one track, but implies governance of the relationship between the tracks.

If we assume that the two tracks report to a single IT manager within a single organization, then vertical line management may provide all the governance you need. But if the two tracks are in separate organizations, then the question of governance becomes a matter for negotiation between the two organizations.

A general governance framework for SOA must support federated, distributed development. Obviously if an enterprise chooses to remain (for the time being) with a simpler development style, then it should not be forced to adopt all the elements of the framework it doesn't yet need. Why should an enterprise adopt principles that are not relevant to the way it is currently doing development? But sometimes it might be correct for an enterprise to adopt principles for federated, distributed development even where the development team is not. For example, to provide some horizontal compatibility with the way that its partners are doing development, or to provide upwards compatibility to the way it intends to do development in future.

Technorati Tags:

Thursday, November 11, 2004

BPEL Usage Patterns

I am currently writing a series of articles on BPEL for the CBDI Journal. In the first of these articles, Orchestration Patterns Using BPEL, I discuss the following six ways in which a company can get benefits from BPEL.

Usage Pattern Description
Process Separation
(e.g. EAI/Workflow)
Process flow logic is contained in a separate layer, which is developed and maintained separately. This may correspond to a technical architecture in which the workflow is executed by a separate software engine.
Process Instrumentation
(e.g. Business Activity Management)
An interface is provided into the orchestration layer, exposing it for direct monitoring and intervention. Now the process flow can be defined and managed by the process manager, often without IT involvement.
Process Componentization
(e.g. grid-like solutions)
BPEL can be used for the orchestration of informal processes, where process components can be assembled rapidly or dynamically as required. This might also support a form of grid-like solution, where process components provide elementary integration scripts.
Process Generalization
(e.g. Application Packages)
BPEL is used to increase the flexibility of an application package, thus allowing it to be implemented in a wider range of organizations with greatly reduced customization.
Process Standardization
(e.g. External Services and Exchanges)
Service-based interfaces using BPEL processes
Process Flexibility
(e.g. Dynamic Outsourcing)
A BPEL-based process allows dynamic changes to orchestration. For example late binding with your service providers, allowing process steps to be easily moved to and fro across the organizational boundary.
 
References
Related posts

Sunday, October 10, 2004

Urgent Process Change

Once we articulate the business process in a separate layer of the SOA, the potential business benefits include crisis management and business continuity.

A business may need to roll out an amended process very quickly, to deal with some crisis. Subject to appropriate controls, we can envisage a company defining and testing a small change to the BPEL script, and then making the new service available for use around the enterprise. The elapsed time from detecting the problem to operating the amended BPEL script might be measured in hours. (Please let me know of any actual benchmarks.)

Of course, an urgent crisis might call for an even faster response than this. There may be a sudden change in the environment, making the current way of doing business uneconomic. There may be a serious security breach. The company is haemorrhaging value, but the prospect of totally shutting down the business process until the problem is fixed is not an appealing one. With BPEL, the process manager should be able to switch over to a previously defined (and thoroughly tested) safe version of the business process (with reduced functionality, increased security, perhaps increased manual intervention, and hopefully reduced vulnerability) to maintain some form of business continuity, in the same manner as reverting to an alternative service.

There is a general point here that we have made before in other contexts. If you want to be confident of flexibility, you need to regularly exercise that flexibility. Don’t wait until you have a crisis before you discover whether your process management is flexible enough. Run the safe version of the process for (say) an hour every month, to make sure it still works.

Sunday, September 19, 2004

BPEL as DSL

Technical Tom shows how we may consider BPEL as a Domain Specific Language. (Update: I have now found a pbblog posting on the same subject from April.) Tom offers an example in which a simple business interaction (withdrawing cash from an ATM) is driven by user-side BPEL rather than supply-side BPEL.

Let's suppose Tom is using a sophisticated smartcard with sufficient memory, processing power and security. It would be theoretically possible to build a smartcard that could run its own BPEL script and send SOAP messages to the ATM. Alternatively, the smartcard could publish its preferred BPEL script, which would then be executed by a BPEL engine inside the ATM. A third option (without smartcard) is that Tom has previously visited the bank's website and edited his own BPEL script, which is now stored on the bank's server.

Meanwhile, the bank has its own business process, together with a fairly strict set of business rules. The actual outcome of the interaction between Tom's smartcard and the bank's ATM needs to be acceptable to both Tom and his bank. If both Tom's process and the bank's process are expressed as BPEL scripts - let's call them BPEL(Tom) and BPEL (Bank) - then we are putting the two scripts together, in other words composing them. Presumably Tom's smartcard will monitor the correctness of this composition in terms of BPEL(Tom).

Of course, the two scripts may be incompatible. BPEL(Tom) may issue some instruction, but BPEL(Bank) may not offer this option at this time. If the two scripts are incompatible, then the composition should of course fail.

Let's suppose that some customers want to withdraw cash regardless of the account balance and accept any charges from a negative balance, while other customers want cash withdrawals to be limited to the available funds in the account. Simply in terms of responding to the variety of demand, it would seem to be a good thing for the bank to offer customers the possibility of tailoring such transactions. In principle, each customer should be free to have as complex a process as he needs, subject to certain reasonable constraints.

So how does the bank enable optimum flexibility for its customers, while maintaining the integrity of its own business process and rules? The challenge on the supply side is to create a platform or environment in which customers can express specific demands, in a way that the supply side can respond to properly. We can think of this as a kind of domain specific language, although not quite in the sense this term is being used elsewhere (including Technical Tom's usage).

Wednesday, September 01, 2004

BPEL

At CBDI, we are generally supportive of BPEL. However, we do not see it as a complete solution to business collaboration. We have raised a number of issues over the past two years.

In business terms, the basis for collaboration between autonomous business entities is a service contract. This suggests a design process in which contracts are paramount - such as Design by Contract. One of our concerns about the web service stack (including BPEL) is that it lacks proper support for contracts and assertions. We proposed that service contract should be a first-class construct, and we have yet to be convinced otherwise. We look forward to seeing what emerges from WS-Choreography.

Secondly, we think the ideal of distribution or federation remains some way off. For example, while WS-Transaction supports distribution, BPEL assumes that the process management is hosted on a single engine, although there have been hints that a future version of BPEL will support distributed process management. In the meantime, some people are experimenting with alternative approaches to distributed process management, such as OGSA.

Thirdly, there appears to be a widespread assumption that the orchestration or choreography is established at design time, with only minor modifications (such as the introduction of new, interchangeable partners) at run time. In time we should progress to real time orchestration.

See CBDI Newswire 22nd May 2003 and 28th May 2003 (free access).

See CBDI Reports From Web Services to Web Collaborations (Nov 2002).

Design by Contract

Design by Contract was pioneered by Bertrand Meyer, who refined the assertion-based approach into the Design by Contract method and the Eiffel language. The basic idea is that a component and its clients have a contract with each other. The client guarantees certain preconditions before calling a method, the component guarantees certain postconditions after the call. If the pre- and postconditions are included in a form that the compiler can check, then any violation of the contract between caller and component can be detected immediately. The prime focus of the approach is to deliver reliable software.

Design by Contract has wide theoretical acceptance. It is an intrinsic component of the Catalysis Approach, perhaps the most influential of all component methodologies. However its usage has to date been relatively limited, the most prevalent use being in the Eiffel language.


In business terms, the basis for collaboration between autonomous business entities is a service contract. This suggests a design process in which contracts are paramount - such as Design by Contract.

For distributed collaborations, a contract between provider and consumer needs to have three parts: logical, quality and commercial.


 
The logical contract specifies the function of the Service, typically using a series of logical assertions such as preconditions and postconditions. This includes both syntactic elements (signature) and semantic elements (vocabulary, meaning).

The quality contract specifies the quality of service - often given the rather dismissive label of non-functional characteristics.

The commercial contract specifies the commercial arrangements, including charging for normal operation and compensation for abnormal operation.

Design by Contract allows services to be defined and used without knowing anything about the implementation. It therefore seems ideally suited to the design of web collaborations. Design by Contract has traditionally concentrated on the logical contract, but the same principles and related design techniques can be extended to the quality contract and the commercial contract.


The term "web service contract" is sometimes used in the very narrow sense of the syntax/signature of the service call, as expressed in WSDL/XML. Much of the recent discussion on Contract-First Development seems to be focused on these aspects of the contract.
  • For example, Simon Guest regards Contract-First Development as an alternative to Data-First Development (which he prefers).
  • See also Aaron Skonnard blog on Contract-First Development (August 2004) with many comments
  • Eric Newcomer (Description First, September 2004) points out that for Web services, the purpose is to share data over the network. Therefore the Schema is more correctly viewed as the contract, not the WSDL.
Design by Contract has a broader concept of contract, which explicitly includes the semantics of the service call in terms of pre/postconditions and invariants, but against a single semantic vocabulary.

Web services will be implemented in a range of formal languages and protocols. These will not only specify the syntax of the service (e.g. using WSDL/XML) but also the semantic vocabulary (e.g. using RDF or DAML-S) and the pragmatics (e.g. using BPEL). However, these languages do not yet cover the full ground of quality contracts and commercial contracts. In the short term, these aspects of the contract are likely to be covered by legal contracts and SLAs between service providers and consumers. Some vendors refer to this broader notion of contract as the Relationship Contract. Some service management tools include such characteristics within the service repository as metadata.


  • CBDI Newswire 22nd May 2003 and 28th May 2003 (free access).
  • CBDI Reports From Web Services to Web Collaborations (Nov 2002) and Modelling for SOA (Feb 2003).

CBDI materials are currently unavailable.

Related posts BPEL (September 2004), Contract First (December 2004)