@RSessions kindly gave me a quick overview of his SIP methodology, and how he calculates the complexity of a system of systems, based on the number of elements in each system and the number of connections between systems. The internal complexity of each system increases in a non-linear manner with the number of elements, and the external complexity increases with the number of connections between the systems, so the trick is to find a structure that optimizes the overall complexity.
Obviously we have to be clear as to what counts as an element (for example functions), and what counts as a connection. Using the SIP lens, it is possible to see how certain architectural styles (including those popular in the SOA world, such as hub-and-spoke or layered) only deliver simplicity (and the benefits of simplicity) if we can assume that only certain kinds of connection are significant. Roger's view is that this assumption is unwarranted and invalid.
In general, the so-called functional requirements are associated with the elements and the logical connections between them. In my view, architects also need to pay attention to the nature of the connections (coupling) because these will have important consequences on the structure and behaviour of the system as a whole. For example, synchronous versus asynchronous. At present, Roger's complexity calculations don't differentiate between different kinds of connection, so it would be interesting to investigate the costs and risks associated with different kinds of connections, to see how much difference it could make.
Roger's primary interest is in IT systems, but the same principles would appear to apply to processes and organizations. If you are running a factory, you have an architectural choice about the connection between say the moulding shop and the paint shop. With an asynchronous flow you have two loosely coupled operations separated by a buffer of work-in-progress; with a synchronous flow you have two tightly-coupled operations connected on a just-in-time basis. The former is a lot easier to manage, but it has an overhead in terms of inventory cost, storage cost, increased elapsed time, slower response to changes in demand, and so on. The latter may be more efficient under certain conditions, but it can be more volatile and the impact is much greater when something goes wrong anywhere in the process.
Intuitively, there seems to be a difference in complexity between these two solutions. The first is simpler, because the connection between the two systems is weaker; the second is more complex. With greater complexity comes greater power but also greater risk. Surely this is exactly the kind of architectural trade-off that enterprise architects should be qualified to consider. Roger's SIP methodology does give the architect a very simple lens to try and understand system-of-system complexity. Not everyone agrees with Roger's definition of complexity, and we can find some radically different notions of complexity for example in the Cynefin world, but at least Roger is raising some important issues. The EA world certainly needs to pay a lot more attention to questions like these.
Saturday, November 28, 2009
Is Enterprise Architecture a Science? Part 2
@RSessions was in London this week, so I sat down with him to continue our previous discussion Is Enterprise Architecture a Science?
The first question to address is - which enterprise architecture are we talking about? I think we both agree that there are some activities within the EA world that look more like religion or mediaeval scholastic philosophy than empirically verifiable science.
For example, in his post What's Right with the Zachman Framework, Grant Czerepak states that "the architectural metaphor conceals what the six perspectives are actually about: Entities, Relationships, Attributes, Constraints, Definitions and Manipulations". And referring to the Kipling-Zachman lens, Grant claims that "the interrogatives have a foundation that goes back over three thousand years across every human culture". In a separate post, he says It's not Aristotle's fault, it's your fault. See my post Arguing with Mendeleev (March 2013).
Lots of EA frameworks are essentially abstract classification schemes that start from an abstract ontological argument ("obviously all businesses are made of objects" or "obviously all processes are made up of nouns and verbs") and make assertions that are not amenable to empirical verification.
Roger's SIP methodology is at least based on an empirically testable (and quantified) hypothesis. That a system of systems with such-and-such measurable structural qualities (in terms of Roger's definition of complexity) will have such-and-such predictable costs. So this provides the basis for a scientifically-grounded engineering practice. I think SIP methodology has a reasonable claim to be scientifically grounded: it can be evaluated not just on whether its prescriptions are practical, cost-effective and useful, but also whether its predictions are true. (Incidentally, I think it would be interesting to compare SIP with Christopher Alexander's early book Notes on the Synthesis of Form. This contains detailed and mathematically grounded work on architectural complexity: although the software gurus who developed structured methods in the 1970s were aware of Alexander's book, they left out most of the difficult detail.)
I think there is a further step before EA could ever dream of becoming a fully empirical science, and this would involve large-scale collection and analysis of empirical data, so that there would be a closed loop between theory and practice, connecting structure and value. In order to achieve this, we should need the active participation of some of the more powerful players in the EA game - the large consultancies and above all the key government agencies that govern IT expenditure. (You know who you are.) At the moment, there is little sign that these organizations are seriously interested in any game-changing innovation. (Roger and I should be delighted to talk to representatives of these organizations, please contact us.)
The first question to address is - which enterprise architecture are we talking about? I think we both agree that there are some activities within the EA world that look more like religion or mediaeval scholastic philosophy than empirically verifiable science.
For example, in his post What's Right with the Zachman Framework, Grant Czerepak states that "the architectural metaphor conceals what the six perspectives are actually about: Entities, Relationships, Attributes, Constraints, Definitions and Manipulations". And referring to the Kipling-Zachman lens, Grant claims that "the interrogatives have a foundation that goes back over three thousand years across every human culture". In a separate post, he says It's not Aristotle's fault, it's your fault. See my post Arguing with Mendeleev (March 2013).
Lots of EA frameworks are essentially abstract classification schemes that start from an abstract ontological argument ("obviously all businesses are made of objects" or "obviously all processes are made up of nouns and verbs") and make assertions that are not amenable to empirical verification.
Roger's SIP methodology is at least based on an empirically testable (and quantified) hypothesis. That a system of systems with such-and-such measurable structural qualities (in terms of Roger's definition of complexity) will have such-and-such predictable costs. So this provides the basis for a scientifically-grounded engineering practice. I think SIP methodology has a reasonable claim to be scientifically grounded: it can be evaluated not just on whether its prescriptions are practical, cost-effective and useful, but also whether its predictions are true. (Incidentally, I think it would be interesting to compare SIP with Christopher Alexander's early book Notes on the Synthesis of Form. This contains detailed and mathematically grounded work on architectural complexity: although the software gurus who developed structured methods in the 1970s were aware of Alexander's book, they left out most of the difficult detail.)
I think there is a further step before EA could ever dream of becoming a fully empirical science, and this would involve large-scale collection and analysis of empirical data, so that there would be a closed loop between theory and practice, connecting structure and value. In order to achieve this, we should need the active participation of some of the more powerful players in the EA game - the large consultancies and above all the key government agencies that govern IT expenditure. (You know who you are.) At the moment, there is little sign that these organizations are seriously interested in any game-changing innovation. (Roger and I should be delighted to talk to representatives of these organizations, please contact us.)
Thursday, November 19, 2009
SOA Planning and Subsidiarity
The principle of subsidiarity states that central planning only applies to those things that require central planning.
Applying this principle during the planning process means finding an appropriate level of consistency and sharing of policy and services and infrastructure. The concept of subsidiarity refers to the scoping level at which a given set of actions and outcomes are coordinated. Which aspects can be (or must be) determined locally, and which aspects can be determined centrally? Which aspects are managed at department level, or at sector level, or at national level? This calls for architectural thinking about management.
In the early phases of SOA adoption, the primary challenge may be to constrain departmental autonomy over some key issues. However, in the later phases of SOA adoption, SOA-based thinking can be used progressively to decouple enterprise activity. Business transformation may sometimes reduce the need for tight synchronization between different processes, while technology and common standards help reduce the interoperability risks. Such changes may have the effect of empowering a degree of greater local autonomy and differentiation (diversity) against a common platform of shared services. This outcome becomes progressively available as the SOA adoption becomes more mature.
Applying this principle during the planning process means finding an appropriate level of consistency and sharing of policy and services and infrastructure. The concept of subsidiarity refers to the scoping level at which a given set of actions and outcomes are coordinated. Which aspects can be (or must be) determined locally, and which aspects can be determined centrally? Which aspects are managed at department level, or at sector level, or at national level? This calls for architectural thinking about management.
In the early phases of SOA adoption, the primary challenge may be to constrain departmental autonomy over some key issues. However, in the later phases of SOA adoption, SOA-based thinking can be used progressively to decouple enterprise activity. Business transformation may sometimes reduce the need for tight synchronization between different processes, while technology and common standards help reduce the interoperability risks. Such changes may have the effect of empowering a degree of greater local autonomy and differentiation (diversity) against a common platform of shared services. This outcome becomes progressively available as the SOA adoption becomes more mature.
Tuesday, November 17, 2009
Knowledge and Architecture
@EnterprisingA (Jon Ayre) suggests an #EAMantra "If you know it, add it to your architecture. If you think you know it, add that too."
My immediate objection to this rule was that it seems to turn the architecture into a kind of brain dump. A good architect knows many things, and if all these items of knowledge go indiscriminately into the architecture, the architecture gets rather cluttered.
@EnterprisingA 's first defence was to say that "the quality of a brain dump depends on the quality of the brain from which the dump comes". And of course only the architect gets to deposit knowledge into the architecture (otherwise there would be a free-for-all).
Unfortunately, the brilliance of the architect is no guarantee that the architect's thoughts all satisfy some meaningful criterion of quality. Intelligence doesn't necessarily mean always having better-quality knowledge, but it should mean having a better capability for processing and developing and filtering and revising knowledge. And as Dumbledore says, "Being - forgive me - rather cleverer than most men, my mistakes tend to be correspondingly huger".
So the architect's knowledge may well get into the architecture eventually, but it doesn't need to go straight in. But @EnterprisingA is (quite reasonably) worried about not writing stuff down. "Keeping it in your head until you are absolutely certain is a good way of never being certain (or being challenged)." "I'd like you to review the arch but it's not certain til it's reviewed so I haven't drawn it so you can't review it..." Of course that's true, but I don't agree that there are only two places to keep knowledge - either in your head or in the architecture. @rgechristie suggests a third possibility - an interim area for in progress work before it hits "the architecture" (that still needs to be accessible by all developers and consumers of the architecture).
***
My second objection was that the architect has a lot of knowledge that doesn't really belong in the architecture at all. There is a strong temptation to put too much into an architecture, and the architectural documents can easily get bloated with miscellaneous material that the architects imagine might be useful to developers and others. (Methodology documents are subject to the same tendency.) For example, the architect may review a design document, may spot a particularly egregious design flaw, and decide to add a rule into the architecture, so that the architecture evolves into a general-purpose design handbook. And if every pearl of wisdom spoken by the architect has to be preserved in "The Architecture", you end up with "The Baroque" - a highly decorated and convoluted style.
It is of course possible to slim down an architecture with too much detail, but in practice this doesn't happen as often as it should. It is much easier to add material to a document than to subtract it, so over time the documents get longer and harder to read.
***
Following our debate on Twitter, @EnterprisingA reformulates his mantra to specify architectural decisions only. "If you know it is the right architectural choice, add it to your architecture. If you think you know it is, add that too." Surely I can't disagree with the proposition that "architectural decisions should be in the architecture", can I?
Actually I do. Sometimes the best architectural decision is to leave something out of the architecture, to leave something deliberately open and unspecified, underdetermined rather than overdetermined. This is a principle of just-enough architecture, or Zen architecture. Sometimes less is more.
My immediate objection to this rule was that it seems to turn the architecture into a kind of brain dump. A good architect knows many things, and if all these items of knowledge go indiscriminately into the architecture, the architecture gets rather cluttered.
@EnterprisingA 's first defence was to say that "the quality of a brain dump depends on the quality of the brain from which the dump comes". And of course only the architect gets to deposit knowledge into the architecture (otherwise there would be a free-for-all).
Unfortunately, the brilliance of the architect is no guarantee that the architect's thoughts all satisfy some meaningful criterion of quality. Intelligence doesn't necessarily mean always having better-quality knowledge, but it should mean having a better capability for processing and developing and filtering and revising knowledge. And as Dumbledore says, "Being - forgive me - rather cleverer than most men, my mistakes tend to be correspondingly huger".
So the architect's knowledge may well get into the architecture eventually, but it doesn't need to go straight in. But @EnterprisingA is (quite reasonably) worried about not writing stuff down. "Keeping it in your head until you are absolutely certain is a good way of never being certain (or being challenged)." "I'd like you to review the arch but it's not certain til it's reviewed so I haven't drawn it so you can't review it..." Of course that's true, but I don't agree that there are only two places to keep knowledge - either in your head or in the architecture. @rgechristie suggests a third possibility - an interim area for in progress work before it hits "the architecture" (that still needs to be accessible by all developers and consumers of the architecture).
***
My second objection was that the architect has a lot of knowledge that doesn't really belong in the architecture at all. There is a strong temptation to put too much into an architecture, and the architectural documents can easily get bloated with miscellaneous material that the architects imagine might be useful to developers and others. (Methodology documents are subject to the same tendency.) For example, the architect may review a design document, may spot a particularly egregious design flaw, and decide to add a rule into the architecture, so that the architecture evolves into a general-purpose design handbook. And if every pearl of wisdom spoken by the architect has to be preserved in "The Architecture", you end up with "The Baroque" - a highly decorated and convoluted style.
It is of course possible to slim down an architecture with too much detail, but in practice this doesn't happen as often as it should. It is much easier to add material to a document than to subtract it, so over time the documents get longer and harder to read.
***
Following our debate on Twitter, @EnterprisingA reformulates his mantra to specify architectural decisions only. "If you know it is the right architectural choice, add it to your architecture. If you think you know it is, add that too." Surely I can't disagree with the proposition that "architectural decisions should be in the architecture", can I?
Actually I do. Sometimes the best architectural decision is to leave something out of the architecture, to leave something deliberately open and unspecified, underdetermined rather than overdetermined. This is a principle of just-enough architecture, or Zen architecture. Sometimes less is more.
Tuesday, November 10, 2009
How Dashboards Work
There are three ways of understanding a dashboard.
Symbolic. A dashboard simplifies and codifies What-Is-Going-On.
Imaginary. A dashboard creates an illusion of an accurate and comprehensive picture of What-Is-Going-On.
Real. The dashboard always falls short of representing What-Is-Going-On. There is always a shadow - something left over that eludes simple capture and representation, often remaining invisible or inaccessible.
Dashboard design and development often focuses on the symbolic - defining the contents of the dashboard as a aggregation of services and data feeds, defining the events that are to be displayed (for example as warning lights), and setting simple policies that set thresholds of attention for predictable events.
Some people seem to view this as a key element of systems thinking. For example Bas de Baar, who blogs under the name "Project Shrink", identifies Systems Thinking As A Technique To Find Project Problems and claims "If I have the right metrics I can ignore everything around me and focus just on the dashboard." He appears to justify this claim by comparing project managers with fish (Swimming Upstream The Information Flow). Fish may be reasonably well adapted to many environments, but they cannot deal with the complex threats posed by the much more intelligent dolphin, as I pointed out in my blog on Lean versus Complex.
However, an emphasis on the dashboard as a simple collection of metrics overlooks the way the dashboard is used within a socio-technical system. Isaac Asimov wrote a story called Reason in which a robot controlled a dashboard perfectly, while refusing to believe in any system beyond the dashboard, but of course that's science fiction.
If we imagine that the purpose of a dashboard is to support prompt and appropriate action in a wide range of normal and abnormal operating conditions, then it should support as much (human, organizational) intelligence as is required to maintain the viability and safety of the system in a given complex environment.
One of the lessons from the Three Mile Island disaster was that when something goes seriously wrong, all the red lights on the dashboard start flashing at the same time, and unless the people in charge of the system have some way of making sense of the emergency (emerging situation), they may make things worse rather than better. (Deming calls this tampering or meddling.)
The safe operation of the nuclear power plant is not just about the design of the dashboard, or about the training of the operators, but about the whole system producing good outcomes in all circumstances. In the Springfield Nuclear Plant, the risk comes not just from idiot operators (Homer Simpson) but from corrupt managers (Montgomery Burns).
Many dashboards are designed merely to aggregate and push information into a social system that the dashboard designer doesn't bother to understand. Prior to Three Mile Island, this was often true even in safety-critical situations. I trust that the safety-critical world has now learned this lesson, but dashboards in other contexts may not be so conscientiously designed and tested.
In management information systems, a dashboard focuses executive attention onto specific aspects of the business. But there seems to be an important difference between a random collection of KPIs or guiding ratios, and a joined-up view that helps the executive reason about the business as a whole system. The standard dashboard is lacking at least two elements of organizational intelligence: Sense-Making (helping the executive see how the different items are interrelated) and Double-Loop Learning (not just getting better at meeting fixed targets, but setting more appropriate targets).
So a dashboard needs to be designed to perform a clear function within an action-based command and control system, rather than merely a simplistic reporting function.
However, there are two traps here. Firstly the hubris of the designer, imagining that a complete understanding of a complex system is possible, or expecting to produce a perfect shadow-free dashboard. Secondly, the blinkered vision of the operator, staring at the dashboard and failing to look out of the window.
In any case, management-by-dashboard seems a pretty unauthentic and disengaged way of running a company. Perhaps it is ironic that HP, one of the leading technology companies of our time, boasts its co-founder Dave Packard as one of the earliest modern practitioners of the opposite technique - "management by walking around". (HP Timeline 1940s)
This blog is an edited version of my contributions to a discussion on the Lenscraft Linked-In group. Thanks to Aidan Ward, Geoff Elliott and Hans Lodder for their stimulating comments.
Related posts: Does Cameron's Dashboard App Improve the OrgIntelligence of Government? (January 2013), Big Data and Organizational Intelligence (November 2018)
Symbolic. A dashboard simplifies and codifies What-Is-Going-On.
Imaginary. A dashboard creates an illusion of an accurate and comprehensive picture of What-Is-Going-On.
Real. The dashboard always falls short of representing What-Is-Going-On. There is always a shadow - something left over that eludes simple capture and representation, often remaining invisible or inaccessible.
Dashboard design and development often focuses on the symbolic - defining the contents of the dashboard as a aggregation of services and data feeds, defining the events that are to be displayed (for example as warning lights), and setting simple policies that set thresholds of attention for predictable events.
Some people seem to view this as a key element of systems thinking. For example Bas de Baar, who blogs under the name "Project Shrink", identifies Systems Thinking As A Technique To Find Project Problems and claims "If I have the right metrics I can ignore everything around me and focus just on the dashboard." He appears to justify this claim by comparing project managers with fish (Swimming Upstream The Information Flow). Fish may be reasonably well adapted to many environments, but they cannot deal with the complex threats posed by the much more intelligent dolphin, as I pointed out in my blog on Lean versus Complex.
However, an emphasis on the dashboard as a simple collection of metrics overlooks the way the dashboard is used within a socio-technical system. Isaac Asimov wrote a story called Reason in which a robot controlled a dashboard perfectly, while refusing to believe in any system beyond the dashboard, but of course that's science fiction.
If we imagine that the purpose of a dashboard is to support prompt and appropriate action in a wide range of normal and abnormal operating conditions, then it should support as much (human, organizational) intelligence as is required to maintain the viability and safety of the system in a given complex environment.
One of the lessons from the Three Mile Island disaster was that when something goes seriously wrong, all the red lights on the dashboard start flashing at the same time, and unless the people in charge of the system have some way of making sense of the emergency (emerging situation), they may make things worse rather than better. (Deming calls this tampering or meddling.)
The safe operation of the nuclear power plant is not just about the design of the dashboard, or about the training of the operators, but about the whole system producing good outcomes in all circumstances. In the Springfield Nuclear Plant, the risk comes not just from idiot operators (Homer Simpson) but from corrupt managers (Montgomery Burns).
Many dashboards are designed merely to aggregate and push information into a social system that the dashboard designer doesn't bother to understand. Prior to Three Mile Island, this was often true even in safety-critical situations. I trust that the safety-critical world has now learned this lesson, but dashboards in other contexts may not be so conscientiously designed and tested.
In management information systems, a dashboard focuses executive attention onto specific aspects of the business. But there seems to be an important difference between a random collection of KPIs or guiding ratios, and a joined-up view that helps the executive reason about the business as a whole system. The standard dashboard is lacking at least two elements of organizational intelligence: Sense-Making (helping the executive see how the different items are interrelated) and Double-Loop Learning (not just getting better at meeting fixed targets, but setting more appropriate targets).
So a dashboard needs to be designed to perform a clear function within an action-based command and control system, rather than merely a simplistic reporting function.
However, there are two traps here. Firstly the hubris of the designer, imagining that a complete understanding of a complex system is possible, or expecting to produce a perfect shadow-free dashboard. Secondly, the blinkered vision of the operator, staring at the dashboard and failing to look out of the window.
In any case, management-by-dashboard seems a pretty unauthentic and disengaged way of running a company. Perhaps it is ironic that HP, one of the leading technology companies of our time, boasts its co-founder Dave Packard as one of the earliest modern practitioners of the opposite technique - "management by walking around". (HP Timeline 1940s)
This blog is an edited version of my contributions to a discussion on the Lenscraft Linked-In group. Thanks to Aidan Ward, Geoff Elliott and Hans Lodder for their stimulating comments.
Related posts: Does Cameron's Dashboard App Improve the OrgIntelligence of Government? (January 2013), Big Data and Organizational Intelligence (November 2018)
Monday, November 02, 2009
Enterprise Architecture and Economics
@jpmorgenthal lays out the reason why enterprise architects should study economics.
But he is not talking about your grandmother's economics. He is particularly impressed by the precise evidence-based insights of the Freakonomics project, and points out the failures of established "best practice" in the economics field.
JP also points out the need for evidence-based decisions to be grounded in the specifics of the problem domain.
I completely agree with JP's demands, but I wonder how many economics courses you could learn these skills from.
JP concludes
So what is the combined market share of EAs with these skills? A tiny fraction of a percent? And what would be the implications of this percentage on IT decisions?
But he is not talking about your grandmother's economics. He is particularly impressed by the precise evidence-based insights of the Freakonomics project, and points out the failures of established "best practice" in the economics field.
"Stephen Levitt undermines what many other Economists, experts and pundits before him rolled out as proven facts and only due to his keen mind and his approach to formulating the problem domain was he able to uncover that which his peers could not."
JP also points out the need for evidence-based decisions to be grounded in the specifics of the problem domain.
"It is the Enterprise Architect's role to ensure that the selection and approaches toward development of systems are sound relative to their business, not just other businesses. Moreover, where decisions are based on the work of other businesses' success, those successes need to be properly vetted to ensure that there is enough commonality between efforts, such that you could make the leap that your business will see similar results. Finally, assumptions and theories need to be tested by properly identifying the variables that need to remain static and then comparing; in essence normalizing the question to be homogenous in all situations."
"You cannot demonstrate the value of Enterprise Architecture if you cannot monetize or enumerate the value of all possible choices relative to the choices that are being recommended or those that have been made. Moreover, it's critical that these analyses are carried out over enough time that short-term wins don't supersede long-term potential gains."
I completely agree with JP's demands, but I wonder how many economics courses you could learn these skills from.
JP concludes
"It is here that Enterprise Architects, especially those we call Chief Architects, truly show their mettle. It is their experience, coupled with the ability to focus on the right set of variables, understanding the impact of change of those variables and being able to communicate that in a way that allows the business to make effective business decisions, which sets top notch practioners apart from Sr. Software Engineers that the organization placated with a title to keep them happy so they wouldn't leave."
So what is the combined market share of EAs with these skills? A tiny fraction of a percent? And what would be the implications of this percentage on IT decisions?
Subscribe to:
Posts (Atom)