Beyond Bimodal

Ten years ago (March 2006) I attended the SPARK workshop in Las Vegas, hosted by Microsoft and inspired by Christopher Alexander. (Every participant received a copy of Alexander's Timeless Way of Building beforehand.) One of the issues we debated extensively was the apparent dichotomy between highly innovative, agile IT on the one hand, and robust industrial-strength IT on the other hand. This dichotomy is often referred to as bimodal IT.

In those days, much of the debate was focused on technologies that supposedly supported one or other mode. For example SOA and SOAP (associated with the industrial-strength end) versus Web 2.0 and REST (associated with the agile end).

But the interesting question was how to bring the two modes back together. Here's one of the diagrams I drew at the workshop.



As the diagram shows, the dichotomy involves a number of different dimensions which sometimes (but not always) coincide.

• Scale
• Innovation versus Core Process
• Different rates of change (shearing layers or pace layering)
• Top-down ontology versus bottom up ontology ("folksonomy")
• Systems of engagement versus systems of record
• Demand-side (customer-facing) versus supply side
• Different levels of trust and security

Even in 2006, the idea that only industrial-strength IT can handle high volumes at high performance was already being seriously challenged. There were some guys from MySpace at the workshop, handling volumes which were pretty impressive at that time. As @Carnage4Life put it, My website is bigger than your enterprise.


Bimodal IT is now back in fashion, thanks to heavy promotion from Gartner. But as many people are pointing out, the flaws in bimodalism have been known for a long time.

One possible solution to the dichotomy of bimodalism is an intermediate mode, resulting in trimodal IT. Simon Wardley has characterized the three modes using the metaphor of Pioneers, Settlers, and Town Planners. A similar metaphor (Commandos, Infantry and Police) surfaced in the work of Robert X Cringely sometime in the 1990s. Simon reckons it was 1993.

Asked "Isn't bimodal new?" ... god no. It's a bad rehash of ideas from a decade or more ago. Even "tri" modal dates back to 1993.

— swardley (@swardley) April 27, 2016

Trimodal doesn't necessarily mean three-speed. Some people might interpret the town planners as representing ‘slow,’ traditional IT. But as Jason Bloomberg argues, Simon's model should be interpreted in a different way, with town planners associated with commodity, utility services. In other words, the town planners create a robust and agile platform on which the pioneers and settlers can build even more quickly. This is consistent with my 2013 piece on hacking and platforms. Simon argues that all three (Pioneers, Settlers, and Town Planners) must be brilliant.

absolutely @mjbrender @richardveryard @TheEbizWizard : all three must be brilliant ... https://t.co/DHMtkmrgzs

— swardley (@swardley) May 4, 2016

Characterizing a mode as "slow" or "fast" may be misleading, because (despite Rob England's contrarian arguments) people usually assume that "fast" is good and "slow" is bad. However, it is worth recognizing that each mode has a different characteristic tempo, and differences in tempo raise some important structural and economic issues. See my post on Enterprise Tempo (Oct 2010).



Updated - corrected and expanded the description of Simon's model.  Apologies for Simon for any misunderstanding on my part in the original version of this post.

---

Jason Bloomberg, Bimodal IT: Gartner's Recipe For Disaster (Forbes, 26 Sept 2015)

Jason Bloomberg, Trimodal IT Doesn’t Fix Bimodal IT – Instead, Let’s Fix Slow (Cortex Newsletter, 19 Jan 2016)

Jason Bloomberg, Bimodal Backlash Brewing (Forbes, 26 June 2016)

Rob England, Slow IT (28 February 2013)

Bernard Golden, What Gartner’s Bimodal IT Model Means to Enterprise CIOs (CIO Magazine, 27 January 2015)

John Hagel, SOA Versus Web 2.0? (Edge Perspectives, 25 April 2006)

Dion Hinchcliffe, How IT leaders are grappling with tech change: Bi-modal and beyond (ZDNet, 14 January 2015)

Dion Hinchcliffe, IT leaders inundated with bimodal IT meme (ZDNet, 1 May 2016)

Dare Obasanjo, My website is bigger than your enterprise (March 2006)

Richard Veryard, Notes from the SPARK workshop (March 2006), Enterprise Tempo (October 2010), A Twin-Track Approach to Government IT (March 2011),

Richard Veryard, Why hacking and platforms are the future of NHS IT (The Register, 16 April 2013)

Richard Veryard and Philip Boxer, Metropolis and SOA Governance (Microsoft Architecture Journal, July 2005)

Simon Wardley, Bimodal IT - the new old hotness (13 November 2014)

Simon Wardley, On Pioneers, Settlers, Town Planners and Theft (13 March 2015)

Lawrence Wilkes and Richard Veryard, Extending SOA with Web 2.0 (CBDI Forum for IBM, 2007)


updated 27 June 2016

Lightweight Enterprise

One of the interesting divisions of opinion at the SPARK workshop (and surfacing afterwards in the blogs of some of the participants) was about something we might call the weight of the enterprise.

Dare Obasanjo asserts My Website is Bigger Than Your Enterprise. And Jeff Schneider retorts My Enterprise Makes Your Silly Product Look Like A Grain of Sand. So who is right?

The basic issue here seems to be the relative amounts of complex engineering required to produce enterprise-grade software versus web-grade software. Dare (and some other participants at the SPARK workshop, including the guys from mySpace) are producing web software with staggering user volumes - much greater than most enterprises have to deal with.

Dare is scornful of enterprise architects, who (he says) "tend to seek complex solutions to relatively simple problems". He and David Heinemeier Hansson are particularly rude about James McGovern.

But as Jeff points out, "most people who have never seen a large enterprise architecture have no concept of what it is". CIOs and enterprise architects generally regard enterprise software (their own realm) as much more challenging than web software, involving much higher levels of complexity, and much stricter "non-functional" requirements (security, robustness, and so on). Are they right, or is this just narrow self-interest?

Size, complexity, weight - these are basic architectural concepts. Part of the difficulty with this debate is that we simply don't have an agreed language for discussing architecture properly. In my opinon, IT education is very deficient in this area. (See Footnote 1.)

Enterprise Mashup

One way of thinking about enterprise mashups is that they provide a way of achieving enterprise-grade systems without enterprise-heavy engineering. Let me give an example of something I regard as a business mashup. My example doesn't involve Ruby, it doesn't involve web services, and many people might say it doesn't even count as SOA. But it has always been my favourite example of the "plug-and-play" business that I describe in my book on the Component-Based Business.

Tesco Personal Finance (TPF) is an example of what I would now regard as a business mashup. It combines (mashes together) banking services (provided by the Royal Bank of Scotland) with retail services (provided by Tesco).

From Tesco's perspective, this mashup provides an extremely lightweight way of entering the banking business. In the old days, if you wanted to open a bank, you had to collect a large amount of gold, which you had to deposit with the banking regulators to satisfy them of your financial stability. Then you had to open a number of branches, and employ a small army of clerks and managers. This represented a considerable investment and risk - which meant that the existing banks could earn high profits behind high entry barriers.

In contrast, TPF represented for Tesco a low-cost, low-risk way of entering the banking industry. Simply plug in some external banking services (which already satisfy all the necessary banking regulations), and a humble retailer can become a bank overnight.

SOA

Although the Tesco Personal Finance initiative predated the SOA and Web 2.0 technologies we have today, we can now interpret it as a precursor or harbinger of these technologies - almost a non-technical proof-of-concept. (See Footnote 2.)

SOA enables the lightweight enterprise. It does this not through services but through architecture. (That, in my view, was the primary justification of SPARK.)

The challenge of architecture is to deconflict the enterprise software space - to create a genuine separation of concerns. This is easier said than done, and it requires some complex architectural thinking. One of the products of this thinking may be platforms on which developers may use small-scale and simple development techniques to produce large-scale and complex solutions.

See my recent posts on the Business Stack (SPARK Workshop 2) and Enterprise Mashups and Situated Software.

---

Footnote 1 - I have always been critical of university IT courses that teach students to write small programs, and fail to teach them the differences between small stand-alone programs and large interconnected suites of programs. If you go to architecture school, you may never build anything larger than a garden shed with your own hands. But you will have looked at skyscrapers and understood the relevant design and construction principles. You won't qualify as an architect without understanding the scale differences between garden sheds and skyscrapers. But in IT there is no such educational requirement. Vast numbers of people get university degrees in IT-related subjects, and even fancy job titles including the word "architect", without having a clue about effects of scale on software.

Footnote 2 - Sometimes technology seems to make new things possible. But we often find that these things were possible all along, they were just too difficult or expensive or risky. In the 1950s, Stockhausen was producing music that predated the modern synthesizer - and it took him months to produce music that later composers would produce with a few quick knob-twiddles. Some might argue that Stockhausen's achievement was all the greater because of his lack of tools. Similarly, many Renaissance painters might have been able to produce their pictures more efficiently if they had had digital cameras. But perhaps their paintings (and their artistic innovations) were all the greater for being done without modern technology. See my post on Art and the Enterprise.

---

Pleasure Principle

In this post, I want to provide some further analysis of Jeff Schneider's useful diagram, which he produced on the first day of the SPARK workshop. (See my blog and his blog.)

Source: Jeff Schneider

 

I interpret the upper half of the diagram in terms of an ecological service design principle I identified in my book on the Component-Based Business. I call this principle the Pleasure Principle. (There is a loose link to Freud's use of the term.)

The Pleasure Principle is about the balance of attention and excitement. Some services provide value (and may experience phenomenal growth) by being exciting, while others provide value (and may experience more steady growth) by being reliable.

I read Jeff's diagram as having high excitement towards the top-left, and low excitement towards the top-right.

High excitement services have "sizzle" and are sometimes called "lean-forward". Low excitement services are "lean-back", and may be designed on the "principle of least astonishment". We may expect entirely different value pricing models for the two types of service. (Economists use a statistical pricing method called Hedonic Pricing, which appears to be a combination of QFD with the pleasure principle.)

Is it possible to have both excitement and reliability at the same time? This is a major question for Web 2.0, and part of the challenge for the supply-side (in the lower half of Jeff's diagram).

I think the answer to this question is an architectural one. We need a stratification which decouples the excitement from the reliability. This is why my view of architecture for SOA and web 2.0 is based on the business stack. See my SPARK notes.

The other big question which hung over the SPARK workshop was the division (true or false) between B2B and B2C. Must we equate the consumer side with high-excitement low-reliability and the corporate side with high-reliability low-excitement? Those (including myself) who see the potential convergence between SOA and Web 2.0 are required to find ways to deconstruct this false division.

One thing that may perpetuate this division, at least in the short term, is the difficulty of turning value into money. (This is sometimes called Monetization, presumably in honour of the painter Monet.) Many service providers differentiate between corporates who are mostly willing to pay, and consumers who mostly aren't. And the best way to get a decent revenue stream in the short term may be to provide a superior (= more reliable) service to the paying customers. But how can we think about the longer-term, without falling into the folly of the dotCom era?

The pleasure principle provides a way of integrating two important elements of service management: service design and service pricing. But the details need more working out. Anyone want to help?

---

Jeff Schneider, A View of Competing Interests and Concerns (19 March 2006)

Wikipedia: Hedonic Pricing, Monet, Pleasure_principle_(psychology), Quality Function Deployment (QFD)

Related post Heartbeat Economy (January 2005) 

Handling Uncertainty

Eighth post in seven days. As regular readers will know, I don't always post this often. But I've got lots of material to share after attending the SPARK workshop, which I hope you find useful and/or thought-provoking. 

One of the most interesting discussions at the SPARK workshop was about business strategy for SOA, with some great contributions from Steve Davis of Disney Studios. The first insight was on handling uncertainty, using Identity Management as an example.

Handling Uncertainty

Who is going to dominate the identity space? There are several plausible scenarios.

• Credit card companies (Visa)
• Government agencies (national ID cards)
• Telecoms companies
• No single dominant position

So what is the appropriate strategic response to this uncertainty?

1. Do nothing until the situation resolves itself.
2. Pick a winning horse. Maybe try to influence the outcome.
3. Develop a strategy that is independent of the outcome.

SOA provides useful support for the third approach. If identity is a significant source of uncertainty (which it is), then abstract away from any identity-related assumptions. Strip all identity from the applications, and prevent developers creating any local identity processing. Establish a standard set of identity-related services, separate from the basic applications. We may then be able to define two different business cases for building and using these shared services.

1. Tactical efficiency. Economics of scale / scope.
2. Strategic flexibility. Ability to accommodate any of the possible scenarios.

Steve argued that four scenarios was a good number - sufficient to provide reasonable variety, while small enough to be meaningful to management.

SPARK workshop 4

Perhaps the coolest diagram that came out of the SPARK workshop was one based on the Celtic Tree of Life.

Nick Gall, who contributed to this model, explains:

This is a recursive and cyclical extension of David Clark's "hourglass" spanning layer model of the Internet. Mike Platt's group had the nice insight that the snake swallowing its tail and the mobius strip are other visual models of two seemingly separate concepts really being a unity. I suppose the Tao symbol is another one.

This is a world in which the word "user" becomes almost useless. (In some domains, the word "user" is a synonym of the word "addict".) This is a world in which consumer/producer relationships are recursive and/or fractal: by consuming we produce, and by producing we consume. mySpace (which was represented at SPARK, and also featured in today's Mix06 keynote) is a great example of this new world.

If I have a problem with the Celtic metaphor, or topological notions like Mobius strips and Klein bottles, it is the implication of closure. The diagram seems to imply a steady-state model, not one that embraces explosive growth based on network externality. mySpace is currently getting over a quarter million new users producer/consumers per day. That's a staggering illustration of the dynamics of Web 2.0.

But one model can't say everything, and the Celtic tree is still a cool model.

Technorati Tags: SOA SPARK SPARK06 service-oriented

SPARK workshop 3

Some more observations from the SPARK workshop ...

Network of centres

I really liked this diagram, which was produced by one of the other groups. I think it echoes some of the ideas of Christopher Alexander about order emerging from the progressive differentiation and completion of a network of centres. The diagram shows a network of services, each drawn as a hub surrounded by some critical Web 2.0 stuff: community, rich content, trust requirements, dynamic mediation. There are centripetal and centrifugal forces, which represent business advantage (outwards) and business constraint/cost (inwards).

Emergence

Nick Gall (Gartner) suggested a simple economic model of emergence, based on benefit to the direct and indirect user, as well as to the provider. This model could help to add richness to the notions of centripetal and centrifugal forces shown in the previous diagram. This reminded me of another model of emergence by Kevin Kelly, which he called Nine Laws of God, included in his great book Out of Control. Kelly's model includes many of the themes that are relevant to innovation in Web 2.0.

Distribute being, Control from the bottom up, Cultivate increasing returns, Grow by chunking, Maximize the fringes, Honor your errors, Pursue no optima; have multiple goals, Seek persistent disequilibrium, Change changes itself.

SPARK 2 - Innovation or Trust?

Day 2 of the SPARK workshop was an attempt to develop frameworks that encapsulated the architectural response to the challenges identified in Day 1. The photoset on Flickr includes some dreadful pictures of me (here with Michael Platt of Microsoft and here with Nick Gall of Gartner), but it also includes copies of most of the flip charts.

The group I was in (which included Glen Harper, Dion Hinchcliffe and Michael Putz) tried to summarize some of the issues around the business stack.



Note the differential rate of change, as well as the gradients of innovation and trust. Note also the questions of horizontal and vertical coupling, which the group discussed but did not resolve.

This is a framework not a fixed solution. For example, in some cases the trust/compliance regime may be stricter (or at least different) at the top of the stack (think healthcare and HIPPA), but in most cases the greatest perceived risks will be associated with the major business assets (legacy) at the bottom of the stack. And (as Anne Thomas Manes reminded us) enterprise innovation isn't always going to be focused exclusively on customer-facing stuff, but may be focused on supply chain or product development or elsewhere.

Different technologies will be appropriate for different levels of the stack - for example we might expect to see SOAP and WS-* at the bottom of the stack (high trust, high engineering) and REST at the top of the stack (low trust, agile).

Of course, stratified architectures and stack diagrams are not new, but they have traditionally been produced from a purely technological perspective (client/service, 3-tier, n-tier computing, and so on). To my mind, the new architectural challenge here is to manage the stratification of layers in a way that responds in an agile and effective way to (the complexity of) the business/user challenges. (Hence Asymmetric Design.)


See also Beyond Bimodal (May 2016)

SPARK workshop

Am in Las Vegas as a guest of Microsoft for the SPARK workshop. (workshop blog, photos). I am very happy to have the chance to meet a load of people whose blogs I read regularly.

We spent the first afternoon on introductions and problem-definition. What is driving change in the architectural environment?

Some of the participants seemed to be primarily interested in rapid development of web applications, using cool new technologies such as Ruby on Rails. For them, the main architectural issues were around handing control over to the users (as individuals or emerging community), and the consequent architectural problems of security and scale.

Other participants (including myself) were interested to see how these themes interacted with business/enterprise themes. Not just web as a platform, but business as a platform. (See John Hagel on Disney, Pixar and Jobs, with my comments.)

Asymmetric Demand went up on the board, at my insistence, although we haven't yet agreed where it belongs.



Jeff Schneider later drew a chart showing the relationship between the user value system and the enterprise value system, which I claimed as a useful illustration of the asymmetry.



So what I see as important, from the first day's discussion, is not just the empowering of users and communities, but also the empowering of the enterprise - the ability to create new kinds of value. The second day is going to be on the architectural response to these challenges. I look forward to it.


Update: Jeff posted a cleaned up version of his diagram in his blog, which I discuss in my post on the Pleasure Principle (March 2006)

Timeless Way

Just been reading Dion Hinchcliffe's post A Timeless Way of Building Software, in which he rewrites the opening of Christopher Alexander's book A Timeless Way of Building into a kind of Web 2.0 manifesto. See my earlier post on Christopher Alexander.

Dion seems to want to present Web 2.0 as something new, and yet grounded in ancient software engineering wisdom. His own examples are all very recent, but his version of Alexander's principles would seem to apply to every piece of software ever produced by Microsoft, especially Windows and Word. As I said in my recent comment Hype and Hyperbola on the impending encounter between Dion and Dare Obasanjo, "technology is full of statements that lack precise empirical support". Especially in the "blogosphere" (stupid word, yes Dare I agree.)

The Google interface is a particularly interesting test case. Much praised for its simplicity, it provides access to some extremely sophisticated computer science - certainly not something you could produce by mashing together a few software design patterns. Nothing timeless about this, unless you count the timeless quality of hard work. And all credit to Joshua (creator of del.icio.us) and the Flickr people for putting in a lot of creative hard work, and getting bought out by Yahoo. You certainly can't attribute their success just to deep study of the Gang of Four book. (Or to Gang of Four song lyrics.)

Coincidentally, the cityofsound blog today provides a couple of extracts from an Observer article on Divine Inspiration, which shows how painter Will Alsop and composer Steve Reich both rely on hard work - doing something that we could call a Focal Practice (as this term is used by the American philosopher of technology Albert Borgman).

Christopher Alexander's original book was called the Timeless Way of Building. The title ends with a verb - a focal practice. Dion Hinchcliffe has stuck a noun on the end, which changes the emphasis completely. And in any case, I think it's the wrong noun.

If there is a timeless way that is illustrated by Dion's examples (and I'm not convinced about this), then it is not about software. It's about business service. As the software increasingly becomes a reconfigurable commodity, then it becomes possible for people to produce new business services - the timeless (but often forgotten) principles and structural patterns here are not about software but about the service economy - supply and demand.

Yet another new blog crossed my path this weekend. In his post Business Patterns, Allan Kelly mentions Christopher Alexander and myself, and points out that I haven't updated my patterns material recently. True - and furthermore I still haven't followed up my post from December 2004 on Patterns and Strategies. I think Allan's line of enquiry may be significantly more productive than Dion's, and I really will try and follow it up this time ...

Christopher Alexander

I've just received a preparation pack for the SPARK workshop, including a new copy of Christopher Alexander's book Timeless Way of Building, first published in 1979. I wonder how many of the SPARK participants have looked at Alexander's later material, including his New Theory of Urban Design (1987), and his magnificent new 4-volume work The Nature of Order.

A few years ago I wrote

Christopher Alexander's work is frequently cited in the software world, usually after a delay of about 15 years. Thus Notes was used by Ed Yourdon and Tom De Marco in the 1970s, to support a view of top-down design. Patterns made a serious entry into the software world the early 1990s. His book on Urban Design has not yet achieved such popularity, although I think it is extremely relevant to business and IT planning. In the past, Alexander has expressed some ambivalence and suspicion about the use of his work by software engineers. More recently, he has been persuaded to make occasional keynote speeches at software conferences and to write prefaces for software books. It may even be true that some software practitioners understand his work better than most architects. However, he is clearly troubled by the fact that software practitioners mostly only pick up fragments of his work, and ignore the holistic aspects of his thinking that he regards as crucial.

 

Here are some of the themes of Alexander's thinking that I see as particularly relevant to SOA.

1. Under the right conditions, complex order emerges from a series of simple steps. A given structure at a given moment is in a partially evolved state.

2. Each step is generally structure-preserving - it builds upon the differentiation and coherence (strength) of the existing structure.

3. Each step brings greater differentiation and greater coherence (strength). Alexander calls this the Fundamental Differentiating Process. (See Nature of Order, Book 2, page 216).

4. Alexander defines structure in terms of a network of centres. Alexander's notions of structure-preservation, differentiation and coherence are explained in terms of this notion of structure. (See "Centers: The Architecture of Services and the Phenomenon of Life," FTPOnline, Richard C. Murphy, March 2004)

5. A key role of governance is then to establish and maintain the conditions under which this fundamental differentiating process can work. (See my articles in the Architecture Journal: Metropolis and SOA Governance and Taking Governance to the Edge).

 

See also Christopher Alexander as Teacher (May 2004), Christopher Alexander 1936-2022 (March 2022)

Hype and Hyperbola

Am still looking forward to the SPARK workshop, despite early posturing from some of the participants. Dare Obasnjo has posted a personal attack on Dion Hinchcliffe, and claims that the Web 2.0 hype makes him feel ill. 

For my part, I wouldn't be remotely interested in going to SPARK if I thought it was just going to be a pleasant gathering of like-minded people. I have politely disagreed with a number of the SPARK participants in my blog before now, and I shall be very bored if there's nothing to disagree about. 

Does Web 2.0 count as hype? To my mind, labels like Web 2.0 (and related concepts such as SOA 2.0 and Business 2.0 - and perhaps even SOA itself) don't refer to a specific technology but to a cluster of characteristic technologies. (Such labels are therefore defined polythetically.) Each of these technologies may follow its own hype curve, but these fine-grained hype curves are probably not in synch. 

 To my mind, the problem with these labels is when people make a fetish of them, and try to draw sweeping conclusions from them. If your notions of SOA, Web 2.0, customer-centricity and enterprise agility are all imprecise, then a statement linking any two of these notions must be regarded as equally imprecise. Technology is full of statements that lack precise empirical support. If we dismiss all such statements as hype, there would be very little left. And if Dare's ears bleed so easily, he must go around with red shoulders all the time.