Data-Driven Data Strategy

One of the things I've been pushing for years is the idea that data strategy should itself be data driven. In other words, if we are claiming that all these expensive data and analytics initiatives are driving business improvement, let's see the evidence, and let's have a feedback loop that allows us to increase the cost-effectiveness of these initiatives. This is becoming increasingly important as people start to pay attention to the environmental cost as well as the monetary cost.

This idea can be found in my ebook How To Do Things With Data and my articles for the Cutter Journal, as well as on this blog.

I doubt anyone will be surprised by Gartner's recent survey, showing that although over 90% of the respondents acknowledged the importance of being value-focused and outcome-focused, only 22% were measuring business impact. So they clearly aren't eating their own dog food.

And the same thing applies to the current hype around AI. Tech journalist @LindsAI Clark asks will we be back in another 10 years wondering who is measuring the business value of all that AI in which organizations have invested billions?

I think we already know the answer to that one.

---

Lindsay Clark, Data is very valuable, just don't ask us to measure it, leaders say (The Register, 21 Feb 2025)

Richard Veryard, How To Do Things With Data (LeanPub) 

Richard Veryard, Understanding the Value of Data (Cutter Business Technology Journal, 11 May 2020)

Wikipedia: Eating your own dog food

 

Business Network Optimization

Some @ATKearney consultants have written an interesting article on Business Network Optimization

"Anyone thinking about rationalizing a network would naturally ask whether so many nodes are really necessary. Networks are a great deal more complicated than that, and managing them requires expansive strategic imagination."

A simplistic accountancy view of a network looks at the direct contribution of each node. From this viewpoint, some nodes may not produce enough direct value to justify their continued existence, and there will be calls for these nodes to be closed or merged with their neighbours.

For example, there are several proposals currently under consideration within the UK National Health Service to rationalize Accident and Emergency provision by closing some hospital departments and relocating staff. These proposals are based on arguments about the optimal size of an Accident and Emergency unit, and on claims that smaller units are unlikely to deliver value for money or clinical  excellence.

Opponents of these closures point to the indirect effect of these closures, including the likely consequences on non-emergency healthcare services at those hospitals that will lack accident and emergency provision, as well as the wider social impact on the local community.

The example given in the A.T. Kearney article is the French postal service, and the authors assert the indirect value of the village post office, using an almost untranslatable French term l'animation du territoire, the "animation of the territory".

The Kearney article identifies three types of business network, which it calls Production, Service and Distribution, and eight elements of network management which must be optimized together. It calls these KNOTs, which stands for Kearney Network Optimization Tools, and asks us not to think of them as merely a laundry list of best practices used to build an optimal network. 



The article illustrates the concept of indirect value in terms of the cross-over between physical and online retailing. If a customer views a product in a physical store and then orders it online, the physical store is providing some indirect value to the retail operation as a whole. It is therefore makes sense to optimize the entire online/offline network as a whole, rather than regarding them as two separate networks. See my post on Showrooming and Multi-sided Markets (December 2012).

Content or Platform?

Yesterday I was talking to a guy from a large media company about its social media strategy. The company is already successfully using social media for distributing and sharing content with its customers, but it doesn't have as much corporate presence as it might like. How might we use social media to talk about ourselves, to let people know about, say, job vacancies or our corporate ethics, he asked.

A media company is judged by its customers according to the range and quality of the content it provides - the company essentially provides a platform for this content. But how do you talk about the platform without interfering with the content? Will customers be turned off if we use the platform to talk about the platform?

At one extreme, there will be customers who really don't want to know about the platform at all. @dgwbirch tweets "I don't want to be friends with my bank, I want to be friends with my bank account. I want to follow my credit card, not my issuer".

But at the other extreme, there will be customers who may value other kinds of interaction. For example, a media company is always swamped with requests from young people for internships and work experience, and we can only grant a limited number of these requests. But suppose we could use social media to give a much larger number of young people the opportunity to develop some skills and show what they are capable of, provide them with decent feedback, and allow them to earn some token of experience that they could mention on their CVs.

Note how an innovative approach to strategy (in this case social media strategy) is driven by a quest - looking for different and innovative ways of creating direct and indirect value within our ecosystem. The widespread success of social media depends significantly on the fact that people are motivated by all sorts of things other than money.

---

When I was a teenager, I entered a DJ competition, for which I had to nominate three singles. I chose one current hit, one old hit, and one new release. I failed to win the competition, and I guessed that my choice had been too obscure for the mainstream. The new release I had chosen was a folksy cover version of a song by Joni Mitchell; several months later, it started to get some airplay and shot to number one. I'm sure there's a lesson there somewhere.

Spending money intelligently

#entarch @MartinHowitt writes "I'm fed up of being told that our IT systems suck. Want better ones? Spend some blinking money. That is all."

@dougnewdick replies "To get better IT you need to spend money, but it doesn't follow that if you spend money you will get better IT."

Obviously just scattering money at random only solves one problem - the problem of having too much money. (When I started in IT, there were some sectors that were hugely profitable and spent vast amounts on IT: although many people imagined that the IT contributed to the profits, the reverse explanation was just as plausible. Correlation is not causality.)

I sympathize with Martin's plea to spend more money, as there is sometimes a tendency to assume that change can be achieved simply by getting people to work harder or smarter, without any investment whatsoever. I suspect that his outburst may have been a response to something like this.

But where is the blinking money going to go? It isn't obvious that the only way to get better IT is to spend money on IT. For example you might possibly get better IT if you spend a bit of money on fresh fruit and fish oil supplements for your IT managers, if that makes them more alert and responsive in negotiations with your IT suppliers, or more empathetc when dealing with staff problems. Or or therapy to help remove unconscious decision-making biases, as @MartinHowitt suggests. I guess many IT architects would say that kind of spending was "out of scope" -- but then what is the scope of IT spending? Surely not just hardware and software?

Which raises an important question for enterprise architecture -- how do we work out the likely consequences of spending money on all kinds of stuff? Shouldn't we be able to compare the benefits of fish oil with the benefits of snake oil?

Good point, says @dougnewdick. "Without proper studies all we can use is experience. But we can ensure investment is connected to desired outcomes."

When I asked what specific aspect of enterprise architecture practice helps to ensures connection of investment to desired outcomes, @fickles replied "All of it - otherwise what's the point?"

I'm sure most enterprise architects would agree with Fred that one of the primary tasks of enterprise architecture is to pay attention to the alignment of effort and outcomes. However, I wonder how many enterprise architects would be willing to take such a broad sociotechnical and humanistic view of this alignment (from snake oil to fish oil), and I wonder what techniques are available to support those who are.

---

See also discussion of Fish Oil on the POSIWID blog.

Complexity and Value - Is Amazon bothered?

@Cybersal complains about Amazon. "Yet again Amazon ships a tiny book with an unreliable courier that my postman could have shoved through my letterbox." and explains "Post office 10 min walk. Courier depot 30 min drive. Courier not a good option for such a low value item."

This raises an important question about Amazon's capability and willingness to manage complexity - specifically differentiation. Amazon already uses multiple alternative delivery channels, but introducing an automatic selection according to the value and size of the item might well further complicate its delivery processes. If Amazon wished to differentiate customers according to the convenience / cost of different delivery channels, it would presumably need to have some geographical reasoning capability. And if Amazon wished to differentiate those products that would fit through a customer's letterbox, it would presumably need to know the size of each customer's letterbox.

We can easily imagine that Amazon could build this kind of capability, perhaps using Google Maps and Google Streetview to check the exact location of Sally's house and estimate the size of her letterbox. But has Sally a right to complain if Amazon doesn't bother?

To what extent do customers have a reasonable expectation of sophisticated differentiation of service - from organizations in general or from Amazon in particular? We may note that most large commercial organizations are way behind Amazon in their ability to enhance customer value through differentiation, while most small commercial organizations are way behind Amazon in their ability to integrate across a complex ecosystem. Because Amazon has already done some amazing things, we may expect it to go even further. But even Amazon has a practical limit of how much complexity it can manage.

We are surrounded by organizations that really can't be bothered, and the value deficit in some sectors is quite disgraceful. So it may be unfair to pick on Amazon, an organization that has done more than most to stretch the limits of the possible. But Sally is right - the future belongs to those organizations able and willing to pay attention to this kind of detail, if it helps to produce direct or indirect value.

Systems of Engagement and Indirect Value

I've just been reading Geoffrey Moore's new white paper Systems of Engagement and The Future of Enterprise IT : A Sea Change in Enterprise IT (AIIM 2011). He calls out a major shift of emphasis in IT, from what he calls Systems of Record (focused on transactions) to Systems of Engagement (focused on interactions). See also presentation by John Mancini and commentary by Jacob Morgan and James Taylor.

Why is this shift necessary? According to Moore, it is because most organizations are dependent upon suppliers or distributors or partners to deliver their fundamental value proposition to their customers. Which means that the new communication and collaboration systems are essential competitive tools. Grab onto them, says Moore, or you will simply end up as roadkill.

(Of course, these challenges aren't just for commercial organizations. Public sector organizations are under just as much pressure to deliver value, even if this pressure is transmitted politically rather than commercially.)

Moore identifies five steps for the CIO to develop systems of engagement for B2B enterprises.

• Make meetings work better across time zones.
• Address complex issues collaboratively.
• Keep collaborators connected for faster decision making.
• Mine community content to extract insights to enhance the business.
• View collaboration and social systems in context.

And five steps for B2C enterprises. Note the strong emphasis on social media.

• Use social media to attract and hold consumer attention. 
• Use social media to extend and improve customer service. 
• Use social media to develop deeper brand relationships and consumer insights. 
• Integrate social media with systems of record to provide a better end user experience.
• Mine metadata to personalize offers for greater relevance and conversion.

I think these lists leave out lots of important issues, but they are a fair start. Clearly, a shift of emphasis from transaction to interaction raises some important questions for the whole enterprise, not just the IT department, which will trigger wide-ranging changes to business operations and governance (from command and control to collaboration).

In his "Confused of Calcutta" blog, JP Rangaswami (now with Salesforce) talks about how Systems of Engagement need to deal with Jyri Engeström's concept of social objects, and explores some of the architectural implications of the granularity of these social objects within the enterprise. (JP also puts in a plug for the Salesforce Chatter platform, which I plan to look at in more detail soon.)

But Moore's thesis about Systems of Engagement only provides part of the story. To take this thinking further, we need to look not just at more collaborative ways of delivering value to customers, but also at the ways in which that value is indirectly experienced by the customers and by the customers' customers and so on. In his paper Creating Value in Ecosystems (December 2010), my friend and associate Philip Boxer argues that "for a supplier to sustain its identity within a dynamic environment ...  it needs to change the way it understands how it creates value, in order to include the indirect forms of value it creates within the larger ecosystem".

Similar ideas have been emerging from the Harvard Business School. In their piece on Shared Value, Michael Porter and Mark Kramer complain about companies "trapped in an outdated approach to value creation", which "view value creation narrowly, optimizing short-term financial performance in a bubble while missing the most important customer needs and ignoring the broader influences that determine their longer-term success". Porter and Kramer believe that the principle of shared value "involves creating economic value in a way that also creates value for society by addressing its needs and challenges" and argue that markets are defined by societal needs, not just conventional economic needs.

Given that organizations must deliver indirect value as well as direct value, the required systems of engagement will take the form of an agile sociotechnical platform to support collaborative composition of value-creating activity. That's a bit of a mouthful, so I'll try and explain it in a future post. Or you could come to my workshop.


See also

• Noah Flowers, Going beyond market-based solutions to create “shared value”(Feb 2011)
• Umair Haque, The New Calculus of Competition (Jan 2011)
• Michael Porter, What is Value in Health Care? (December 2010)
• Michael Porter, Discovering - and Lowering - the Real Costs of Health Care (2011)

From SOA to Better Judgement

In the comments to my post From Complex Events to Predictive Analytics, I have been debating with Hans Gilde about the causes of the credit crisis, and the possibility that better systems and technologies might have prevented the crisis.

Hans writes

"if people had not deluded themselves into ignoring obvious risks, they would have put more realistic calculations in their spreadsheets, resulting in better judgment that would, AFAIK, have prevented the credit crisis"

Hans is not alone in making this kind of assertion, which appears to be largely based on an optimistic belief in the power of technology to solve problems. But my understanding of economics is that it is a pessimistic (or "dismal") science, in which periodic crises are pretty much unavoidable. Like forest fires or domestic arguments, they can be delayed or suppressed for a while, but that simply makes them bigger when they eventually break out. Certain politicians claimed that "we will never return to the old boom and bust" but, as we now discover, presided over a much larger boom and bust [Channel Four News]. The idea that there is any technocratic solution to fundamental economic problems seems like hubris.


I am also worried by Hans' appeal to "better judgement". Of course we'd all like to think that SOA produces better judgements, so I tried looking for evidence of this, but an internet search for "SOA" and "better judgement" mostly found pages in which the words "better judgement" were preceded by the words "against" or "despite". Oh dear.

But even if SOA does produce better judgements, there is still a problem: better for whom? The service-oriented world is essentially a distributed one, with no central judge, no supreme court. So there is no fixed standard of value.

There was an interesting example just yesterday on a BBC Radio programme called More or Less. Paul Wilmott (described by the BBC as a "financial mathematics guru") explained how a typical bonus system motivates financial traders to copy one another in order to maximize their expected bonus, even though this inhibits portfolio diversification and therefore reduces the overall stability of the bank. So what counts as a better judgement for the trader does not necessarily count as better for the bank and its customers. If you provide better information to the trader, this may well help the trader maximize his bonus, but doesn't necessarily produce better results for anyone else. (The argument can also be found on Paul's blog, in a post called Science in Finance V: Diversification.)

Of course it's easy enough to see what the problem is here - it is that the bonus system is wrong. But if you design computer system improvements without considering the inevitable imperfections in human systems, you are very likely to get a nasty shock.

In any case, even if you had a bonus system that perfectly aligned the motivation of the trader with the interests of the bank, and even if you could produce a computer system that perfectly calculated the True Value of every asset in the world, traders wouldn't use these calculations because they wouldn't generate enough money. What traders really want is a recursive system that calculates what every other trader thinks the value is, in order to spot market movements a few moments before everyone else. But why would anyone expect the widespread possession of such systems to make the global economy any less unstable?

There are many possible positive contributions that SOA might make to the operation and supervision of complex trading systems. But if we want these systems to produce good results for the right people, we have to do some really hard systems thinking, and not just hope that speeding things up is going to solve all the problems.

Utility Computing and Profitability

In this month's interview [GigaOM Interview, March 10, 2008], Om Malik pushed Ray Ozzie to say something about the profitability of serving the cloud.

"It’s unlikely that we would get into it if we didn’t think it was going to be a profitable business. So we’ll just manage it to be profitable. It’s going to have different margins than classic software, or the ad (-supported) business. But, we have every reason to believe that it will be a profitable business. It’s an inevitable business. The higher levels in the app stack require that this infrastructure exists, and the margins are probably going to be higher in the stack than they are down at the bottom." (2008)

Let me repeat what Ray says here: "We'll just manage it to be profitable." It is certainly conceivable in some markets that a single dominant player can determine how the added value is shared between the layers in the stack. In our work on service-oriented business strategy, we look at some of the ways of influencing the value ladders in a complex stratified business ecosystem. So has Microsoft really got the power to exert that kind of hegemony over the Internet Cloud? Maybe not, but it certainly looks like it's going to try.

Economic Rent

Can the business case for SOA be based on the concept of economic rent? I'd be interested to get some reaction to the following preliminary discussion ...

Economists use the term "rent" to mean something different to the everyday meaning of the term. It is not the rental income generated by renting out an asset (such as land) but the market power associated with a given market position. For example, let's say that an executive would work for $250,000 per year, but manages to negotiate a salary of $1,250,000 per year. The difference between these two figures ($1m) is a measure of his market power, and this is what economists refer to as economic rent. Economic rent is also earned by professionals with some barrier to entry (such as doctors and lawyers) and by celebrity performers (such as musicians and sportsmen).

There are two alternative definitions of economic rent, yielding somewhat different calculations. In classical economics, it is defined as the difference between the actual income and the necessary income. For example, imagine a popular musician who currently earns around £50,000 per concert. If his popularity waned, his earnings might drop to around £10,000 per concert, but he would continue to play. The difference between these two figures is pure excess - economic rent.

Followers of Pareto use a slightly different definition - the difference between the actual income and the next best opportunity. For example, if our musician wasn't playing concerts, his next best activity might be as a session musician at £4000 per week.

John Kay writes: "economic rent arises from differentiation and migrates to wherever in the value chain scarcity is found". Kay illustrates this with The Story of Champagne. Under perfect market competition, there are no clusters of market power, and no economic rent - prices are adjusted until the level of supply exactly equals the level of demand.

With software and services, of course, the potential income sometimes bears no relationship whatsoever to the costs of production. Most of the profits of Google or Microsoft can be regarded as economic rent in this sense. Some people regard these profits as excessive, while others would justify such profits as reward for past risks and initiatives taken; in any case, the continuation of these profits depends on maintaining some exclusive advantage, something that is difficult for others to replicate.

If a service provider is to invest in developing a new service, then this may be done with the expectation of receiving sufficient reward to cover the (mostly front-loaded) costs and risk. The level of value generated by a service depends on its exclusivity and differentiation: if the service would be easy to copy then there is little prospect of economic rent; but if the service gains a strong foothold, then it may be difficult to dislodge, at least in the short term.

It is for this reason that network effects are so important to the economics of SOA. A service with ten million consumers delivers greater value to each consumer than a similar service with only ten thousand consumers. This advantage enables the provider of the more popular service to extract an economic rent.

Another important consideration is the relationship between fixed development costs (sunk costs?) and ongoing operational costs (opportunity costs?). (If I understand the Wikipedia article correctly, this relationship is one of the areas where classical economics and paretian economics diverge.) SOA and other forms of virtualization (including grid) make it easier to shift costs from fixed to variable - but of course this may not deliver the best economic advantage to the service provider.

If you have some market power, how do you calculate the best possible price? There is some tricky mathematics involved here: let's assume that the more you charge, the fewer users you have, and let's assume that there is a non-zero cost of servicing each user. Under certain conditions, the maximum rent is given by a formula known as Hotelling's Rule.

In case you were wondering, Hotelling has nothing to do with hotels. Harold Hotelling was a mathematical statistician, also known for Hotelling's Law ("in many markets it is rational for producers to make their products as similar as possible") and Hotelling's Lemma (relates the supply of a good to the profit of the good's producer).

Wikipedia: Comparative Advantage Economic Rent, Hotelling's Law, Hotelling's Rule

Value Conflicts

A topical political row in the UK between church and state reveals some underlying issues for the service economy.

The UK Government has decided that the adoption process should not discriminate against gay couples, and relevant legislation will come into force in April. But a portion of the adoption process is outsourced to faith-based agencies that are not willing to provide such services to gay couples, and these agencies have requested special exemption from the legislation. (Protests have been led by Catholic adoption agencies, with the support of the Roman Catholic hierarchy in the UK, but other Christian leaders have also expressed their support.) [BBC News]

I do not wish to comment on the issue itself. I want to discuss some more general structural questions.

Firstly, we might look for a way that the Catholic agencies could continue to provide some valuable services in this area, that is compatible both with their religious beliefs and the prevailing legislation. It might be possible to decouple the contentious parts of the adoption process (selection?) from the non-contentious elements (support?), and find ways for the Catholic agencies to provide the non-contentious elements in a mutually acceptable way.

This would presumably involve changes either to the way the adoption process is decomposed into services, and the way these services are funded, or to the ways in which different agencies collaborate with one another to deliver a complete set of services.

This raises a second important question. If a service provider has a strong religious or ethical objection to X (whatever X may be) then we obviously don't expect that service provider to provide services that perform X. But what about delivering services into a larger composite system that happens to include X?

Thus the Catholic adoption agencies may be faced with a difficult choice. Either to provide some useful services, even though these services may be used within a larger system that includes elements the Catholic agencies don't like. Or to refuse to provide these services unless they are able to control the end-to-end process according to their own values - perhaps in the hope that their protest will result in some larger change in public policy.

Note that we don't always know how our services may be mashed-up or reused (repurposed). P.G. Wodehouse once wrote a story in which Bertie Wooster and his friends ran a gambling syndicate on the length of sermons in country churches. Presumably the clergymen were not aware of this syndicate, and would have been horrified to find out. But they probably wouldn't have stopped delivering sermons just because some people were abusing them.

On the internet, of course, you may be able to monitor how people are using your services, and close down or inhibit inappropriate uses. But how feasible is this, given that it is sometimes difficult to take any action without causing problems for legitimate users?

The more general point is that an open network of services will almost always involve heterogeneous value systems - whether this is a secular public sector working with faith-based voluntary agencies, or IT vendors working with an increasingly complex ecosystem. Just yesterday for example, Chandler Howell discussed Apple's growing difficulties imposing secrecy on its partners [Secret Apple Sauce].

These are (among other things) structural problems. The SOA architect may not be in a position to resolve value conflicts, but may be able (in collaboration with others) to produce structures that can operate effectively and efficiently despite the (inevitable) value conflicts. Open or closed.

Technorati Tags: SOA service-oriented