Beyond Bimodal

Ten years ago (March 2006) I attended the SPARK workshop in Las Vegas, hosted by Microsoft and inspired by Christopher Alexander. (Every participant received a copy of Alexander's Timeless Way of Building beforehand.) One of the issues we debated extensively was the apparent dichotomy between highly innovative, agile IT on the one hand, and robust industrial-strength IT on the other hand. This dichotomy is often referred to as bimodal IT.

In those days, much of the debate was focused on technologies that supposedly supported one or other mode. For example SOA and SOAP (associated with the industrial-strength end) versus Web 2.0 and REST (associated with the agile end).

But the interesting question was how to bring the two modes back together. Here's one of the diagrams I drew at the workshop.



As the diagram shows, the dichotomy involves a number of different dimensions which sometimes (but not always) coincide.

• Scale
• Innovation versus Core Process
• Different rates of change (shearing layers or pace layering)
• Top-down ontology versus bottom up ontology ("folksonomy")
• Systems of engagement versus systems of record
• Demand-side (customer-facing) versus supply side
• Different levels of trust and security

Even in 2006, the idea that only industrial-strength IT can handle high volumes at high performance was already being seriously challenged. There were some guys from MySpace at the workshop, handling volumes which were pretty impressive at that time. As @Carnage4Life put it, My website is bigger than your enterprise.


Bimodal IT is now back in fashion, thanks to heavy promotion from Gartner. But as many people are pointing out, the flaws in bimodalism have been known for a long time.

One possible solution to the dichotomy of bimodalism is an intermediate mode, resulting in trimodal IT. Simon Wardley has characterized the three modes using the metaphor of Pioneers, Settlers, and Town Planners. A similar metaphor (Commandos, Infantry and Police) surfaced in the work of Robert X Cringely sometime in the 1990s. Simon reckons it was 1993.

Asked "Isn't bimodal new?" ... god no. It's a bad rehash of ideas from a decade or more ago. Even "tri" modal dates back to 1993.

— swardley (@swardley) April 27, 2016

Trimodal doesn't necessarily mean three-speed. Some people might interpret the town planners as representing ‘slow,’ traditional IT. But as Jason Bloomberg argues, Simon's model should be interpreted in a different way, with town planners associated with commodity, utility services. In other words, the town planners create a robust and agile platform on which the pioneers and settlers can build even more quickly. This is consistent with my 2013 piece on hacking and platforms. Simon argues that all three (Pioneers, Settlers, and Town Planners) must be brilliant.

absolutely @mjbrender @richardveryard @TheEbizWizard : all three must be brilliant ... https://t.co/DHMtkmrgzs

— swardley (@swardley) May 4, 2016

Characterizing a mode as "slow" or "fast" may be misleading, because (despite Rob England's contrarian arguments) people usually assume that "fast" is good and "slow" is bad. However, it is worth recognizing that each mode has a different characteristic tempo, and differences in tempo raise some important structural and economic issues. See my post on Enterprise Tempo (Oct 2010).



Updated - corrected and expanded the description of Simon's model.  Apologies for Simon for any misunderstanding on my part in the original version of this post.

---

Jason Bloomberg, Bimodal IT: Gartner's Recipe For Disaster (Forbes, 26 Sept 2015)

Jason Bloomberg, Trimodal IT Doesn’t Fix Bimodal IT – Instead, Let’s Fix Slow (Cortex Newsletter, 19 Jan 2016)

Jason Bloomberg, Bimodal Backlash Brewing (Forbes, 26 June 2016)

Rob England, Slow IT (28 February 2013)

Bernard Golden, What Gartner’s Bimodal IT Model Means to Enterprise CIOs (CIO Magazine, 27 January 2015)

John Hagel, SOA Versus Web 2.0? (Edge Perspectives, 25 April 2006)

Dion Hinchcliffe, How IT leaders are grappling with tech change: Bi-modal and beyond (ZDNet, 14 January 2015)

Dion Hinchcliffe, IT leaders inundated with bimodal IT meme (ZDNet, 1 May 2016)

Dare Obasanjo, My website is bigger than your enterprise (March 2006)

Richard Veryard, Notes from the SPARK workshop (March 2006), Enterprise Tempo (October 2010), A Twin-Track Approach to Government IT (March 2011),

Richard Veryard, Why hacking and platforms are the future of NHS IT (The Register, 16 April 2013)

Richard Veryard and Philip Boxer, Metropolis and SOA Governance (Microsoft Architecture Journal, July 2005)

Simon Wardley, Bimodal IT - the new old hotness (13 November 2014)

Simon Wardley, On Pioneers, Settlers, Town Planners and Theft (13 March 2015)

Lawrence Wilkes and Richard Veryard, Extending SOA with Web 2.0 (CBDI Forum for IBM, 2007)


updated 27 June 2016

Twin-Track Architecture

#entarch This post follows discussions with Graham Berrisford of Avancier about the relationship between Enterprise Architecture (EA) and Solution Architecture (SA).

---

What seems to make sense is to describe EA/SA as a twin-track process (similar to the twin-track process that aligns service development with solution development in SOA).

Twin-track is essentially an abstract process pattern, used to align two activity streams at different levels of granularity. As far I recall, I first encountered it in the Select Perspective methodology, described in two books by my friend and former colleague Paul Allen. The CBDI SAE methodology also uses the twin-track approach.

In SOA, there is a stream of activity to produce and maintain services, and another stream of activity to use these services to build solutions. These two streams need to be connected but not synchronized. The two tracks may be separated organizationally - split into different teams or org units, or even separate companies - provided that there are suitable governance mechanisms for allocating resources and responsibilities, and resolving issues. People may specialize in one or other stream.

A twin-track processes doesn't assume either top-down or bottom-up. In some cases, you could start with the solution requirements and then specify the desired services (top-down). In other cases, you could start with a set of available services, and then assemble these into solutions (bottom-up). In practice, the twin-track approach accommodates a mixture of top-down and bottom-up. The important point however is that the two have to be connected at a series of touch-points.

We can now describe the relationship between enterprise architecture and solution architecture in similar terms. If solution architecture is disconnected from enterprise architecture, then the solution architects are likely to produce point solutions instead of enterprise solutions. Conversely if enterprise architecture is disconnected from solution architecture, then it is likely to produce grand, simplistic and ultimately unimplementable schemes. Again, we don't have to assume that either EA or SA is dominant, but that they work together towards common goals.

---

For other posts on twin-track: browse, subscribe. 

A Twin-Track Approach to Government IT

#ukgovit @instituteforgov has just published a report called System Error: Fixing the Flaws in Government IT.

The report recommends a twin-track approach to government IT, based on the two concepts of Agile and Platform.

"The platform must standardise and simplify core elements of government IT. For any elements of IT outside the platform, new opportunities should be explored using agile principles. These twin approaches should be mutually reinforcing: the platform frees up resource to focus on new opportunities while successful agile innovations are rapidly scaled up when incorporated into the platform."

The report acknowledges the tension between these two concepts ...

"Treating items as commodities reduces cost but can limit flexibility; coordinating elements of IT across departments frees up resources but may move them further from frontline users; common standards support interoperability but also restrict the freedoms to innovate."

... and offers some general ideas for managing this tension.

• To act fully in the interests of government, an agile approach requires a light touch form of coordination at a system level. 

• To minimise duplication of effort in solving the same problems, there needs to be system-wide transparency of agile initiatives. 

• Existing elements of the platform also need periodic challenge. ... Transparency, publishing feedback and the results of experiments openly, will help to keep the pressure on the platform for continual improvement as well as short-term cost savings.

Trouble is, some of this stuff is really hard. The report talks glibly about "a less than intelligent customer", referring first to business users having an inadequate conception of the possible, and then to the public sector as a whole lacking the collective knowledge and skills to negotiate effectively with suppliers. This lack of intelligence is apparently blamed on the V-model development process, which creates the impression that the adoption of Agile methods would solve this problem. But the idea of Agile as a silver bullet is a dangerous one, as many people have already pointed out on the Linked-In discussion group.

One way of understanding the twin track approach is to think of the different kinds of economics involved.

• 'Platform' means delivering economies of scale and economics of scope.
• 'Agile' means delivering economies of alignment.

Combining the two introduces some complex architectural challenges, as I've written about here and elsewhere before. We call this Asymmetric Design. For an example of this approach applied to public sector IT, see an analysis of the CSA Case by Philip Boxer and myself. See also The Impact of Governance Approaches on SoS Environments (pdf) by Philip Boxer and others.


In the current economic situation, the public sector as a whole is charged with making massive cost savings, and it is crazy to imagine that cost savings of this scale would not be associated with significant structural change, including IT systems. This kind of disruptive innovation goes way beyond the economies of scale and scope, and introduces some serious questions about the economics of alignment.

The word "architecture" is mentioned a few times in the Institute for Government report, but only in passing as something that the Government CIO will look after. (Mostly technology or solution architecture, I only found one single reference to business architecture.) So there is an implicit idea of central thinking and hierarchical governance. But there are some architectural challenges here that are some way beyond the current practices of enterprise architecture.

Governance is also a significant problem. The report comments on the pendulum swings between centralized and decentralized provision, which is something we noted in the CSA case, and was also present in the case of ContactPoint (which we were in the middle of writing up when it was cancelled). Such pendulum swings are often a characteristic symptom of weak or unsustained governance.

Not only is this stuff structurally complicated, but there are some commercial stakeholders that have every incentive to maintain the complicated status quo, thanks to a grossly dysfunctional procurement process.

And there is an even bigger problem with the report, which is that it looks at government IT exclusively from within government - in other words, from the perspective of civil servants. For example, the report adopts a supply-side notion of "joined-up government", understood largely in terms of internal linkages and efficiencies between systems, and fails to mention the demand-side notion of "joined-up government" that involves a coherent experience for the citizen. (See my post on Joined-Up Government from December 2005.)

Meanwhile the notion of "user" appears to refer mainly to civil servants and other public sector workers. Surely the purpose of government IT is not to provide direct value to civil servants but to provide various forms of indirect value to individual citizens and socioeconomic communities.

The report regrets that "government IT [is] falling further and further behind the fast-paced and exciting technological environment that citizens interact with daily" and indicates "the potential for IT ... fundamentally changing the relationship between citizen and state". "Around the world governments are using technology to help them deliver better services, be more transparent and accountable, and connect more directly with their citizens." (Examples are cited from Canada, USA and Malaysia.)

And yet the report fails to explain how "agile" can adequately represent the demand side requirements of citizens, interacting with a broad range of government services while going about their public business. There is a completely different notion of "platform" required here - government as a platform, which Tim O'Reilly and others have been talking about for a couple of years. And a different notion of agility, which goes a lot further than agile software development.

---

Other commentary

See Linked-In discussion group

Harry Metcalfe (2 March 2011) observed that many of the recommendations in the report were really hard, and was one of the first to complain about the insufficient attention to procurement in the report.

Software Product Lines

A little while ago, a colleague pointed me to a presentation called Synergies between Service-Oriented Architecture and Software Product Lines (pdf) by Christoph Wienands of Siemens Corporate Research.

This presentation refers to my work on Business Modelling for SOA, as well as a couple of papers by my Everware-CBDI colleague John Butler.

As Wienands reminds us, the SEI notion of Software Product Lines involves a form of twin-track development. One track produces software products, while the other track produces "core assets". This is very similar to the twin-track idea that SOA has inherited from component-based software engineering (CBSE).

Wienands gives a sketch of some software variability within Siemens - especially between the European operation (using SAP) and the US operation (using PeopleSoft). He mentions some of the dimensions of this variability, including ontology (different data models) as well as technology (different platforms). This kind of variability (and how we deal with it under SOA) is something I've talked about a lot in my Business Modelling articles and workshops.

---

I am not longer affiliated with the CBDI Forum. The CBDI Journal archive can be found on the Everware-CBDI website. http://everware-cbdi.com/cbdi-journal-archive

Twin-Track Governance

Twin-track development involves a management separation between the development of components and services, and the development of larger artefacts that use these components and services. It was a characteristic feature of the more advanced CBSE methods, such as Select Perspective, and is obviously relevant to service engineering as well.

Clarification Update: Select Perspective has always been more than just a CBSE method, and now qualifies as a full-fledged SOA method. Even as a CBSE method, it was one of the first to embrace service as a first class construct. Although this may not be immediately obvious from the Select website, it is clear from an article contributed by Select consultants (then part of Aonix) to the CBDI Journal in 2002.

The twin-track pattern can be superimposed on a traditional lifecycle process, such as that recently propagated by OASIS for the web service lifecycle (pdf). Even though the OASIS process is presented as applying to a single web service, it doesn't take much reframing to see this kind of process applying to an artefact that is larger than a single service - such as a whole system or subsystem - provided it is specified and designed in one place at one time. So we can start to see the OASIS process (or a suitably generalized version of it) applying to either of the twin tracks - but not at the same time. Both tracks may be following some version of the OASIS process, but they don't talk to each other.

The twin-track pattern is sometimes interpreted in a simple way, with an IT organization divided into a service-creation track and a business project track. The service track produces services with web service interfaces; the business projects produce applications with user interfaces (UI). In this interpretation, use of BPEL belongs exclusively in the service track, because it doesn't produce applications with UI.

However, we can interpret twin-track in a much more powerful way than this, by generalizing the pattern. We simply specify that supply of services is in one track, and the consumption of these services (even if this is in the context of using BPEL to build larger artefacts that are themselves services) is in another track. The point of the twin-track pattern is that the supply and consumption can be decoupled and managed separately - possibly even in separate organizations. Of course, this pattern can be applied more than once, yielding more than two tracks in total.

Meanwhile, the possession of a UI is probably of secondary importance here. With SOA, we can (and probably should) build applications that give the user a choice between interacting directly or via some user-built secondary application. Thus for example, I want my online bank to offer me a set of services rendered in two alternative ways: firstly via a UI (probably browser-based) and secondly as a series of webservices (or equivalent) that I can invoke from within some desktop money management program. Think of Google and eBay delivering the same services via browser and via web services. In the service economy, I want all interfaces to have something like a webservice or REST or RSS/Atom alternative.

And perhaps if lots of people are using desktop money management programs, it might be cheaper for the bank to give all remaining customers a low-functionality desktop program (or recommend a suitable bit of freeware) and then decommission the UI altogether. I'm not saying this is always going to be advisable, but it's certainly an option. So we might see a growing number of serious business applications with no traditional UI at all.

In architecture, if you build windows everywhere, it makes it harder to join buildings together. Think of a university campus, which grows piecemeal over many decades. If every new block has a blank wall, then it is easier to build another block next to it. If you put windows on every available wall, you have to put useless space between the blocks, and then build silly walkways to save people keep going down to the ground floor. The evolution of complex SOA raises some of the same issues.

Even with single-track development, there is a governance question here. How can we maintain order over a complex and evolving system, if we cannot simply outline all the requirements at the beginning? And with twin-track, a critical function of SOA governance is governing the relationship between the tracks, however many tracks there may be.

How do we get (and keep) all these loosely coupled development processes and operations processes in alignment with each other, and also in alignment with the business. And alignment with IT accounting (financial and otherwise) would be nice too. Obviously the whole point of twin-track is that you can decouple the tracks to some extent, but if you decouple them too much then you throw baby (reuse, interoperability, flexibility) out with the bathwater (agile=fragile).

Some sources advocate frequent synchronization between development teams. While synchronization does not necessarily rule out federated, distributed development, but this would only be possible with a great deal of horizontal coordination. And this would introduce lots of other challenges.

SOA governance governs what kind of development is appropriate, and how it should be coordinated. For example, SOA governance provides ground-rules for participants to agree interfaces before they go off and do their own thing, and for enforcing these agreements. In the real world, we know that all agreements are subject to being reneged and renegotiated as requirements and conditions change. But who incurs this risk, and who shall bear the cost of any change? If you decide you need to change the interface, am I forced to respond to this, and if so how quickly?

Change management (e.g. avoiding uncontrolled demand for service specification change) cannot be managed within either one track, but implies governance of the relationship between the tracks.

If we assume that the two tracks report to a single IT manager within a single organization, then vertical line management may provide all the governance you need. But if the two tracks are in separate organizations, then the question of governance becomes a matter for negotiation between the two organizations.

A general governance framework for SOA must support federated, distributed development. Obviously if an enterprise chooses to remain (for the time being) with a simpler development style, then it should not be forced to adopt all the elements of the framework it doesn't yet need. Why should an enterprise adopt principles that are not relevant to the way it is currently doing development? But sometimes it might be correct for an enterprise to adopt principles for federated, distributed development even where the development team is not. For example, to provide some horizontal compatibility with the way that its partners are doing development, or to provide upwards compatibility to the way it intends to do development in future.

Technorati Tags: governance SOA service-oriented

A Brief History of Methods

In this blog posting, I am going to apply a knowledge management framework called Cynefin to a series of Business/IT challenges: software development, systems development and service development.

Background

The Cynefin Centre spun off from IBM in July 2004 and describes itself as a network focusing on the application of complexity science to management and organisational practice. "At the heart of the Cynefin Centre is a distinction between ordered and unordered systems, and the consequent recognition that systems with fundamentally different qualities require the application of contextually differentiated methods for both diagnosis and intervention."

The Cynefin Sensemaking Framework has five domains, four of which are named, and a fifth central area, which is the domain of disorder. The right-hand domains are those of order, and the left-hand domains those of un-order.

For a full description of the framework, see paper by Cynthia Kurtz and Dave Snowden: The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal Vol 42 No 3, 2003 (html) (pdf). See also weblog by Willem van den Ende (Dec 6, 2004) (Jan 17 2005).

Development of Methods

Chaos	Uncoordinated building work No method
Known	Design of a single software system Methods include RUP and XP.
Knowable	Design of a software-intensive solution, as a system of systems (e.g. RUP/SE) Twin-track/ multi-track development (e.g. Select Perspective) Assumption of overall design authority - directed composition.
Complex	Design of a software-intensive experience, within a service-based ecosystem - true SOA. Complex systems engineering calling for collaborative composition. Methods could include XB.

RUP is the flagship method for IBM Rational. There is a plug-in for systems engineering called RUP/SE, which goes some way towards the demands of SOA.

There is a difference of opinion as to where extreme Programming (XP) belongs. If it is merely a software engineering method focused on the efficient production of small-scale software, then it belongs in the domain of the known. If it moves beyond software productivity into business agility, then it transforms into extreme Business (XB) and belongs in the domain of the complex.

Related Posts

RUP/SE (January 2005)
The Authorship of Method (February 2011)

The Planning Dilemma

One of the key problems faced by planning (in IT and elsewhere) has been the dilemma – top-down or bottom-up. Top-down methods produce grand schemes without addressing the problems on the ground (including legacy), while bottom-up methods produce local solutions without any overall order, coherence or reuse.

Bottom-Up Approach (Point Projects)	Top-Down Approach (Area Projects)
Local short-term initiative. No mandate to pay attention to broader, longer-term opportunities and effects.	Broader, longer-term initiative
Building a solution against immediate requirements (where “building” means design, construct or assemble)	Focus on system properties across a whole area (e.g. business domain, technical domain, infrastructure)
Strongly aligned to local objectives. Direct link between (local) benefits, costs and risks.	Indirect links between benefits (across area), costs and risks Often difficult to create/maintain business case for adequate investment in resources and infrastructure Often difficult to demonstrate return on investment
Cost-effective use of conveniently available resources (improvisation or “bricolage”)	Creating value by establishing (procuring or building) conveniently available resources

One way of addressing this dilemma is to introduce a twin track process, involving a top-down stream of activity and a bottom-up stream of activity.

Obviously for this twin-track process to be effective, we need clear allocation of responsibility, authority, expertise and work (RAEW). This is an aspect of governance - making sure the right things are done in the right way. Twin-track development exposes the inevitable tensions between business goals and service needs. And in federated/distributed development, these tensions are replicated across multiple business entities; governance then becomes a question of negotiation between two separate organizations, rather than simple management resolution within a single right framework.

---

This is a modified extract from an article on Business-Driven SOA published in the CBDI Journal, June 2004. For further extracts from this article, please see my Slideshare presentation on Organic Planning. See also our papers in the Microsoft Architecture Journal on Metropolis and SOA Governance (July 2005) and Taking Governance to the Edge (August 2006).

The notion of twin-track development is included in the Practical Guide to Federal SOA, published by the CIO Council in 2008. See also

• Stuart Charlton, On Effective Architecture (October 2007)
• Jeevak Kasarkod, CBDI SAE Reference Framework Continues to Evolve (InfoQ November 2010)
• Dave Mayo, Agile IT (June 2011)
• Mike Papazoglou, Engineering Services (??)
• Ronald Schmelzer, Why Service Consumers and Service Providers Should Never Directly Communicate (Zapthink, Feb 2008) with my response on Agility and Variation (March 2008)
• David Sprott, Beware the new silos (March 2011)

See also: What does Top-Down mean? (September 2011)

For other posts on twin-track: browse, subscribe.