Organizational Intelligence - Does Size Matter?

On a Linked-In group discussion (CBDI Forum), Richard Gilyead asked some excellent questions about organizational intelligence, and I'm going to post an edited and expanded version of the discussion here.

---

Richard started by asking

Does size matter? Do organisations with "thin skin" have a better ability to sense and respond? Is it inevitable that organisations will develop "thicker skins" as they grow?

Of course it has long been a popular idea that large organizations should behave like small organizations, while small organizations have problems of their own. I believe we can use organizational intelligence as a "lens" to attain a more precise understanding of some of the things that get lost as an organization gets larger, and use this lens to try and preserve and restore these elements. Meanwhile, a small organization is more dependent than a larger one on the knowledge and intelligence of its business partners and other members of its ecosystem, and it is useful to pay explicit attention to these aspects of its external relationships.

Richard then asked a follow-up question

What aspects of cohesion should be emphasised to preserve small organisation benefits as they grow? 

I think this should be a vital question for enterprise architecture (even though enterprise architects mostly work in large organizations, it's surely never to late to ask it), and I don't think there is a one-size-fits-all answer.

An organization makes a strategic choice (whether consciously or unconsciously) about the kind of "structural coupling" that matters. For example, a high-tech start-up company may be physically located near a university campus, and the founders have good personal relationships with researchers at the university. This will help the organization remain close to technological trends, but possibly at the expense of developing a broader understanding of market opportunities. At some stage the venture capitalists may persuade them to move their head office closer to the customers and/or to bring in new executives whose personal network faces in a different direction. Meanwhile, a different company might be dominated by supply chain and distribution issues, and for that company it will be these relationships that are most strategically significant.

The point is that this "structural coupling" affects many aspects of organizational intelligence: the events and trends that the organization can respond to, knowledge flows, innovation and organizational change. Which brings us back to the question of stability - where are the fixed points around which organizational systems (including formal information systems and services) can be built, and where are the points where flexibility is required. This is not a new question, but the organizational intelligence "lens" provides a new way of thinking strategically about the implications here.

Scale and Governance

Following my post Does Britain Need Smaller Banks? Fred Fickling objected to my point that large banks are more difficult to regulate.

"Not sure that the simple answer here is really the right one. Size & corp responsibility aren't necessarily inversely related."

Of course, some large companies can have a highly refined sense of ethics and social responsibility, although this doesn't always protect them from strong disagreements with external lobbyists, as Shell discovered when it wished to decommission the Brent Spar.

In any case, not all large companies can be trusted to regulate their own behaviour in the public interest. Companies pay tax where it suits them, move (or threaten to move) operations from one country to another, shrug off massive fines, and can sometimes behave as if they were above the law, as Fred concedes. In the long term, these legacy corporations may be doomed, but in the short term they can still cause a lot of disruption in the financial ecosystem.

Of course there are no simple answers. I certainly think some companies are too large, and I hope there will always be good opportunities for smaller companies to compete. What I'm calling for is a way of reasoning intelligently about the size of companies, which balances the interests of all stakeholders in a fair and governable manner. I think this is a proper topic for business architecture. And perhaps some of what we've learned from SOA about granularity and governance can be applied to this greater problem domain.

Does Britain Need Smaller Banks?

George Osborne, possibly the next Chancellor of the United Kingdom Exchequer, has asked some good questions about the right size of banks. (Robert Peston, Tories to break up banks?, BBC 8th April 2009)

"We cannot allow one part of our economy to behave in a way that puts the rest of the economy at risk when it fails. We need to think deeply about whether we can sustain banks that are not only too big to fail, but potentially too big to bail."

"We should look at whether Britain in fact needs smaller banks. For it would be a bitter irony if we came out of this crisis with a banking system that was even more concentrated and even riskier than the one we had before it."

It is no longer obvious that large banks are any more financially viable than small banks. What seems pretty clear is that large banks are more difficult to regulate. What was the economic logic behind all those mergers and acquisitions in the banking sector anyway? Whatever may have been gained in terms of the economies of scale were totally blown away by the economies of governance.

Umair Haque asks (rhetorically) why bankers never talk about reforming banks. However, as Osborne points out, now that the British government owns a large chunk of the banking sector, there is an opportunity to make radical changes to the sector as a whole, rather than merely controlling the behaviour of each company within the sector. I hope this will allow a serious debate on the scale and granularity of business. See my earlier post on Business Model and Financial Viability.

Update: Here's what Henry Mintzberg has to say about General Motors, in a piece called America's monumental failure of management (Globe and Mail, 16 March 2009).

We hear now about "too big to fail." "Too big to succeed" is more like it. General Motors has been going slowly and painfully bankrupt for decades, managerially as well as financially. The new money will only put off its demise. Americans will have to face this reality sooner or later.

See also Towards Partition (Economic Principles, April 2010).
and Do banks need more intelligence? (Demanding Change, May 2010)

Business Model and Financial Viability

A venture capitalist called Fred Wilson goes back to basic economics - When Talking About Business Models, Remember That Profits Equal Revenues Minus Costs. As Phil Wainewright comments "Has the world become so blind to the basics of commerce that it needs reminding of such a basic tenet? Apparently yes."

Fred is certainly not saying that all companies should try to survive on current revenues alone. Clearly there will be many companies that will continue to attract investment based on future projected revenue - that's exactly where the venture capitalist comes in. However, the viability of a start-up company is entirely dependent on the willingness of people to invest their time, energy and money against some future returns.

What Fred is saying is that these returns are ultimately based on profit rather than revenue. Start-up companies can no longer afford to employ large number of people, if that merely results in a slightly better product and slightly higher revenues. What matters to the venture capitalist are such ratios as operating leverage - how much projected revenue per employee.

Leverage rather than size. Bigger isn't necessarily better. In the recent past, many mergers and acquisitions have been justified in terms of financial engineering rather than genuine creation of value. Fred's point is that even organic growth is suspect if it doesn't make the organization more viable. Why on earth does Facebook (for example) need a thousand employees? What value are they actually going to produce?

A few years ago, we had people asking What is the Right Size of a Service? Now I think the industry is ready to ask the next question: What is the Right Size of a Service-Oriented Business? Many organizations oscillate unstably between uncontrolled growth and painful cost-cutting. It would be better to have a more stable approach to questions of size and viability, and the business model needs to help us with these questions: What is the ideal size of a given business, and how can this size be maintained?

---

Update: See Seth Godin on The Right Size

Blueprints 2

Reposting my comment to Nick Malik's post on Blueprints, following my previous post on Blueprints. Nick asked me to spell out the implications of my previous comment - whether I meant that we don't need accuracy, or that we should start measuring - and challenged the strong association I implied between accuracy and measurement.

Firstly, let me affirm that I think measurement (in the broadest sense) is a good idea.

I am not sure I know what accuracy means except in terms of measurement. How can we reason about things like "structural integrity" and "quality" without some form of measurement? In engineering, we don't generally expect perfect integrity or perfect quality (which is usually either physically impossible or economically non-viable); we look for arguments of the form "X produces greater integrity/quality than Y" - where X/Y is some choice of material or technique or pattern or whatever. So there are implicit metrics running through all branches of engineering. Software engineering just isn't very good (and should be much better) at managing these metrics and making them explicit. As a result, we don't always see software engineers delivering the maximum integrity and quality for the minimum cost and effort.

So when I'm talking about measurement, I'm certainly not only interested in cost estimation and other project management stuff. I think architects should be thinking about things like the amount of complexity, the degree of coupling, the scale of integration, and you certainly can't read these quantities straight from a UML diagram.

Of course a building blueprint doesn't tell you everything you need either. If you are designing an airport, the blueprint will show how much flooring you need, but will not show whether there is enough space for the passengers to queue for passport control. If you are designing a tower block, you have to have some way of working out how many lifts to put in. In software engineering this kind of stuff is dismissed as non-functional requirements.

All engineering involves estimation. "is this bridge going to fall down" is an estimate.

In a traditional waterfall development, many people thought it was a good idea to address the functional requirements first (logical design), and then tweak the design (physical design) until you satisfied the non-functional requirements as well. But when you are composing solutions from prebuilt enterprise services, this approach just doesn't wash. Indeed, it may now be the other way around: if a service assembly fails some functional requirement, you may be able to plug/mash in some additional service to fill the gap; but if it fails the non-functional requirements you may have to throw the whole thing away and start again.

Finally, I don't say only big projects need accuracy. If a government builds a tiny service to be used by the entire population, a small project might have a massive impact. A garden shed may not need a professional architect: that's not because a garden shed doesn't need accuracy, but because an amateur builder can work out the quantities accurately enough herself.

Blueprints

Nick Malik has prompted a great discussion on the difference between accuracy in architecture and IT. He asks why IT architects don't produce blueprints that are as accurate as those produced by architects in the traditional world of physical construction.

The kind of accuracy Nick describes in traditional architecture is about quantity. The costs of a building are largely determined by the physical dimensions. (The cost of the carpet depends on the floor area.) So the first person who looks at the blueprint is not the builder but the quantity surveyor. The blueprint has to be good enough to enable reasonably accurate cost estimation.

We don't usually do that in IT. There is no How-Many/How-Much column in the Zachman framework. You can't work out quantities from a UML diagram. In a pre-SOA world, we thought cost estimation was largely about counting the number of components to be constructed (simple, medium, complex) and putting them into a time/effort formula. But this approach to cost-estimation is increasingly irrelevant to SOA.

If you are only building a garden shed then you possibly don't need a professional architect or surveyor. If you are building a tower block then you certainly do. The people who are doing serious architecture in an SOA world are those operating at internet scale - for example redesigning Skype so that it doesn't fall over on Patch Tuesday (see Skype Skuppered).

Scale and Self-Organization

Werner Vogels, CTO of Amazon, knows a thing or two about implementing large-scale distributed SOA. So it's worth taking seriously when he tells us that

"For any truly scalable agile environment, self-organization is essential."

In his post Reading References, he goes on to recommend some papers and books on scalability and self-organization.

• Richard Conniff, The Limits of the Alpha Male
• Steve Grand, Creation: Life and How to Make It

My own favourite book on self-organization remains Kevin Kelly's book Out of Control, now available online on KK's website. See also the entry on Self-Organization in Principia Cybernetica.

If we accept that self-organization can be effective for addressing some classes of complex problem, next step is to develop practical methods for engineering self-organizing systems. There are some promising research projects in this area, and there's a conference in Leibzig in September (SOAS 2007). I probably won't have time to attend the conference myself, but I shall look out for the proceedings. I wonder whether Amazon will be represented?

See also Werner Vogels on Scalability (April 2006)

Fractal Loading

Fractal loading means that each high-level exchange also carries with it simultaneous exchanges on many smaller levels, and implies the coexistence of different but related things at different levels of scale.

The opposite case of monofunctional planning forces many separate and competing exchanges of the same type into a single communications channel, thus maximizing the capacity of uniform communications channels dedicated to a single type of exchange. An example of this is a choked highway, or the overloading of subway cars at rush hour. Not only is this inefficient, but it excludes other types of exchange.

[source: Information Architecture of Cities, Coward and Salingaros]
See also Phil Jones wiki

Customer relationship management illustrates the alternative between fractal loading and monofunctional planning. A call centre or other customer-facing operation may aspire to identify additional products and services to sell to customers. But this conflicts with a series of aspirations related to the efficiency and speed of a single transaction type – e.g. maximum throughput, minimum transaction times, and minimum queueing time.

Management-by-walking-around (MBWA) is an appeal to fractal loading. Knowledge management and trust benefit from fractal loading.

Fractal loading represents a major challenge to traditional bureaucratic assumptions about information processing and management. It helps us understand why traditional approach can never deliver adequate levels of adaptability.

---

Originally posted at http://www.veryard.com/notions/2004/06/fractal-loading.htm 
or http://www.users.globalnet.co.uk/~rxv/notions/2004/06/fractal-loading.htm