Bruce Schneier praises A Rational Response to Peanut Allergies and Children.
Many schools attempt to enforce a peanut ban. My son isn't allowed to take peanut butter sandwiches for lunch. In software terms, this is equivalent to perimeter security.
A better way of protecting children with allergies of all kinds is to ensure that the children and their carers are careful, and that they can deal promptly with any accidental ingestion. This is more robust, because it is not dependent upon total reliability of the perimeter defence. It is also more flexible, because it deals with a broader range of allergies (not just peanuts) without insisting on exclusion of every possible allergen.
In software terms, this is equivalent to deperimeterization.
Wednesday, January 28, 2009
Monday, January 26, 2009
Cloud Computing or what?
In previous posts, I have contrasted SOA with JBOWS (just a bunch of web services) and CEP with JBOCE (just a bunch of complex events).
I am now happy to announce a new acronym to contrast with Cloud Computing: JSOWD (just a shower of water droplets). I did have a ruder version but I thought it was a bit early in the morning.
Seriously, we are now facing the usual semantic confusion. On his blog, J.P. Morgenthal explains How To Define SOA and Cloud Computing. He is not offering a definition as such, merely suggesting the kind of work that would be required to produce a satisfactory definition.
I certainly don't want to get bogged down into debates about what Cloud Computing is "all about" - or even the negative debate about what it's not all about. Like most other bits of jargon, cloud computing involves a loose bundle of characteristics, and it is unlikely that everyone will agree as to which of these characteristics are essential or even desirable.
Which is where a schema comes in. A good schema should help practical people reason about the dependencies between real-life problems, specific technology characteristics, and potential benefits, as well as sharing practical knowledge. John's customers don't want to know what abstract labels and dogmas to attach to their projects, but I guess they do want practical and specific guidance. (If I can put in a plug for the CBDI Forum, this is the approach we have taken with SOA. See the CBDI Forum SAE Model.)
I am now happy to announce a new acronym to contrast with Cloud Computing: JSOWD (just a shower of water droplets). I did have a ruder version but I thought it was a bit early in the morning.
Seriously, we are now facing the usual semantic confusion. On his blog, J.P. Morgenthal explains How To Define SOA and Cloud Computing. He is not offering a definition as such, merely suggesting the kind of work that would be required to produce a satisfactory definition.
"Instead of trying to define them in a narrative manner, I believe we need to define them as a scheme with many components that are interrelated."In a comment to JP's blog, John Evdemon writes
"Let the pundits debate their concepts and bizarre acronym versioning schemes - I'd rather help customers solve their problems."
I certainly don't want to get bogged down into debates about what Cloud Computing is "all about" - or even the negative debate about what it's not all about. Like most other bits of jargon, cloud computing involves a loose bundle of characteristics, and it is unlikely that everyone will agree as to which of these characteristics are essential or even desirable.
Which is where a schema comes in. A good schema should help practical people reason about the dependencies between real-life problems, specific technology characteristics, and potential benefits, as well as sharing practical knowledge. John's customers don't want to know what abstract labels and dogmas to attach to their projects, but I guess they do want practical and specific guidance. (If I can put in a plug for the CBDI Forum, this is the approach we have taken with SOA. See the CBDI Forum SAE Model.)
Sunday, January 25, 2009
SOA and Holism
Wikipedia's definition of holism traces back to Jan Smuts
In my writings I've drawn on the recent work of Christopher Alexander, from A New Theory of Urban Design to The Nature of Order, where Alexander talks about something he calls structure-preserving transformations.
According to the New Theory (or at least my interpretation of it, which I call Organic Planning) each act of transformation should be a step within a larger and open-ended evolutionary development, and should have three aspects.
I published a very simplified version of this in the CBDI Journal in 2004, suggesting that the service designer needed to look in four directions (upwards, downwards, sideways, inwards). I understand that this was picked up and referenced by the ArchiMate people, for example in a 2005 book called Enterprise Architecture at Work, and it has been implemented in the Telelogic tool. My colleague Tony Bidgood has recently published an article on ArchiMate, with some further examples.
1. Inwards: Functional correctness
2a. Downwards: Integrating and composing smaller stuff
2b. Sideways: Interoperability
3. Upwards: Larger whole
Organic Planning is described in my 2001 book on the Component-Based Business, and there is a short version on my website, but I didn't make this explicit in my 2004 article.
My expectation has always been that a series of transformations (whether structure-preserving or otherwise) would be expressed as a series of model pairs, in which the nth TO-BE becomes the (n+1)th AS-IS. But some alternative modelling notations (such as Michael Bell's SOMF) allow both AS-IS and TO-BE to be expressed in a single model, so it would be very interesting to see how a series of transformations could be expressed and analyzed.
Holism ... is the idea that all the properties of a given system (biological, chemical, social, economic, mental, linguistic, etc.) cannot be determined or explained by its component parts alone. Instead, the system as a whole determines in an important way how the parts behave.To what extent does this concept apply to SOA? My own view is that SOA needs to be understood from the Systems-of-Systems Engineering paradigm rather than from the Systems Engineering or Software Engineering paradigm. This helps us to deal with a range of system level phenomena including Feature Interaction.
In my writings I've drawn on the recent work of Christopher Alexander, from A New Theory of Urban Design to The Nature of Order, where Alexander talks about something he calls structure-preserving transformations.
According to the New Theory (or at least my interpretation of it, which I call Organic Planning) each act of transformation should be a step within a larger and open-ended evolutionary development, and should have three aspects.
- Produce something at some level
- Complete something (larger) that was already part-developed - typically by linking smaller (lower-level) things and peer (same-level) things that already existed, or were created in previous steps.
- Create new opportunities - Alexander calls this hinting-at.
I published a very simplified version of this in the CBDI Journal in 2004, suggesting that the service designer needed to look in four directions (upwards, downwards, sideways, inwards). I understand that this was picked up and referenced by the ArchiMate people, for example in a 2005 book called Enterprise Architecture at Work, and it has been implemented in the Telelogic tool. My colleague Tony Bidgood has recently published an article on ArchiMate, with some further examples.
- Richard Veryard, Business Driven SOA (CBDI Journal, May 2004), Business Driven SOA Governance (CBDI Journal, June 2004)
- Tony Bidgood, Archimate Standard for Enterprise Service Modelling (CBDI Journal, August 2008)
1. Inwards: Functional correctness
2a. Downwards: Integrating and composing smaller stuff
2b. Sideways: Interoperability
3. Upwards: Larger whole
Organic Planning is described in my 2001 book on the Component-Based Business, and there is a short version on my website, but I didn't make this explicit in my 2004 article.
My expectation has always been that a series of transformations (whether structure-preserving or otherwise) would be expressed as a series of model pairs, in which the nth TO-BE becomes the (n+1)th AS-IS. But some alternative modelling notations (such as Michael Bell's SOMF) allow both AS-IS and TO-BE to be expressed in a single model, so it would be very interesting to see how a series of transformations could be expressed and analyzed.
See also SOA and Holism 2 (February 2009)
Saturday, January 24, 2009
Explaining SOA to a Martian
Suppose you were explaining pottery to a Martian. You’d have to explain what a pot was, and what it was for. Pots are used for a variety of purposes, including the storage and serving of liquids (such as wine and oil), the display of cut flowers, and cookery. You’d have to say that one of the main characteristics of a pot was its ability to hold water.
Then you’d start to describe the process: The potter scoops up some wet clay from the bottom of a river; she slaps it around, spins it and shapes it, before sticking it in an oven. ‘Hold on’, says the Martian, ‘Do you really expect me to believe that a potter, however skilled, can make a waterproof pot out of wet clay?’
And when you put it like that, of course, it is hard to believe. If we hadn’t seen it done with our own eyes, we’d start to doubt it possible.
In the days of component-based software engineering, we were often presented with an equally implausible story. A software engineer scoops up some wet COBOL from a legacy system, slaps it around a bit, wraps a few shells around it, and voila! - a fully interoperable, WS*-compliant business object.
Champions of SOA are faced with a similar challenge. A complete story of SOA has to contain at least four things.
I'm not saying you have to explain the intricacies of software engineering or service engineering to your Martian (or businessman). But you do have to have some broadly plausible story. Because without a decent structure, a decent process, and decent materials, the value ain't gonna happen.
Then you’d start to describe the process: The potter scoops up some wet clay from the bottom of a river; she slaps it around, spins it and shapes it, before sticking it in an oven. ‘Hold on’, says the Martian, ‘Do you really expect me to believe that a potter, however skilled, can make a waterproof pot out of wet clay?’
And when you put it like that, of course, it is hard to believe. If we hadn’t seen it done with our own eyes, we’d start to doubt it possible.
In the days of component-based software engineering, we were often presented with an equally implausible story. A software engineer scoops up some wet COBOL from a legacy system, slaps it around a bit, wraps a few shells around it, and voila! - a fully interoperable, WS*-compliant business object.
Champions of SOA are faced with a similar challenge. A complete story of SOA has to contain at least four things.
- A story about structure. Not just the structure of a single service, but the whole architecture. This is like explaining the shape of the pot.
- A story about process. How do these services get produced and consumed? How does the model-driven, contract-first approach actually work? What are the roles and responsibilities in an organized SOA approach?
- A story about raw materials. What are services made out of? Lumps of software? Lumps of executable model? BPEL or OSLO?
- A story about value. Who can extract value from these services, and how?
I'm not saying you have to explain the intricacies of software engineering or service engineering to your Martian (or businessman). But you do have to have some broadly plausible story. Because without a decent structure, a decent process, and decent materials, the value ain't gonna happen.
Creating Value by Informed Governance
These days, small bookshops only survive if they can beat Amazon. My local bookshop (Pitshanger Books, in West London) consistently does. I ordered a book on Enterprise Architecture last Saturday, and it arrived on Monday lunchtime. (Despite the database telling us that there wasn't a copy anywhere in the country. Lesson 2: never trust the database.) Thanks Walter!
So now let me tell you about the book.
by Martin Op't Land, Erik Proper, Maarten Waage, Jeroen Cloo, Claudia Steghuis.
Springer 2009
This book is intended as an introductory textbook for masters-level students or new practitioners of enterprise architecture (EA). This book presents Enterprise Architecture as an instrument, positioned between strategy and programme management, to join strategy formulation with strategy execution.
Enterprise face certain challenges to their survival.
The book gives a useful overview of some of the popular frameworks for enterprise architecture - notably Zachman, ArchiMate and TOGAF 8.1 - although these frameworks are presented fairly uncritically. It also provides some helpful process guidance. What does the architect need to do, and what are the required competencies and desired personality of the architect?
But the point of enterprise architecture is to provide stakeholders with insight into organization change (p23). But what is the nature of this insight? Where should enterprise architects focus their attention, and how do these frameworks and processes help them to pay attention to the right things? In particular, if agility is the top concern these days, how does an enterprise architect reason about agility?
The book acknowledges several open issues in the field of enterprise architecture, including the lack of cost/benefit analysis for the practice of EA (p129, p133). But there is a broader need for quantification - reasoning about the value and viability of the enterprise as a whole. Most of the models used by enterprise architecture are essentially line-and-box diagrams, with no facility for quantification, so that even important architectural judgements about cohesion and coupling are subjective and unreliable. Martin Op't Land's doctoral thesis (Applying Architecture and Ontology to the Splitting and Allying of Enterprises, 2008) does address some of these issues, but this material is not included in the book under review.
Overall, this is a short and well-written introduction, covering a lot of useful material, but leaving plenty of important topics for the following books in the series. ...
So now let me tell you about the book.
Enterprise Architecture: Creating Value by Informed Governance
by Martin Op't Land, Erik Proper, Maarten Waage, Jeroen Cloo, Claudia Steghuis. Springer 2009
This book is intended as an introductory textbook for masters-level students or new practitioners of enterprise architecture (EA). This book presents Enterprise Architecture as an instrument, positioned between strategy and programme management, to join strategy formulation with strategy execution.
Enterprise face certain challenges to their survival.
- Keep up or perish
- Comply or bust
- ...
The book gives a useful overview of some of the popular frameworks for enterprise architecture - notably Zachman, ArchiMate and TOGAF 8.1 - although these frameworks are presented fairly uncritically. It also provides some helpful process guidance. What does the architect need to do, and what are the required competencies and desired personality of the architect?
But the point of enterprise architecture is to provide stakeholders with insight into organization change (p23). But what is the nature of this insight? Where should enterprise architects focus their attention, and how do these frameworks and processes help them to pay attention to the right things? In particular, if agility is the top concern these days, how does an enterprise architect reason about agility?
The book acknowledges several open issues in the field of enterprise architecture, including the lack of cost/benefit analysis for the practice of EA (p129, p133). But there is a broader need for quantification - reasoning about the value and viability of the enterprise as a whole. Most of the models used by enterprise architecture are essentially line-and-box diagrams, with no facility for quantification, so that even important architectural judgements about cohesion and coupling are subjective and unreliable. Martin Op't Land's doctoral thesis (Applying Architecture and Ontology to the Splitting and Allying of Enterprises, 2008) does address some of these issues, but this material is not included in the book under review.
Overall, this is a short and well-written introduction, covering a lot of useful material, but leaving plenty of important topics for the following books in the series. ...
Friday, January 23, 2009
Joined-up Policing
Here's another possible example of joined-up service (see Joined-Up Healthcare and Six Business Improvements).
According to a story in the Australian Courier-Mail (November 2008), the Queensland Police Service and the Queensland Police Union are currently in dispute about the effectiveness of a new records management system called QPRIME, supplied by Canadian firm Niche RMS (via Bruce Schneier).
The police officers complain that data entry, including navigating the new system, takes far too long. According to Mr Leavers, vice-president of the Union, said the system turned jobs that usually took an hour into several hours of angst.
A spokesman for the police authority claims that the problems are caused by unfamiliarity with the new system.
I don't want to criticize QPRIME based on a single newspaper report, but the report hints at a much more general issue with complexity and business improvement.
This kind of complexity is neither produced nor resolved by SOA technology alone. The challenge for the business analyst is to understand the possible interconnections between multiple instances, to design services that render these interconnections properly, and to develop a workflow that allows users to access these services efficiently and effectively.
According to a story in the Australian Courier-Mail (November 2008), the Queensland Police Service and the Queensland Police Union are currently in dispute about the effectiveness of a new records management system called QPRIME, supplied by Canadian firm Niche RMS (via Bruce Schneier).
The police officers complain that data entry, including navigating the new system, takes far too long. According to Mr Leavers, vice-president of the Union, said the system turned jobs that usually took an hour into several hours of angst.
"There was an occasion where two people were arrested on multiple charges. It took six detectives more than six hours to enter the details into QPRIME," he said. "It would have taken even longer to do the summary to go to court the next morning, so basically the suspects were released on bail, rather than kept in custody."
A spokesman for the police authority claims that the problems are caused by unfamiliarity with the new system.
"The benefits of the QPRIME system into the future far outweigh short-term disaffection by some officers. It is already showing its worth in assisting officers to solve significant crimes by allowing them to access information in a holistic manner."There seems to be some agreement that QPRIME works well for simple incidents. The problem seems to arise when there are multiple suspects for multiple crimes. As one police officer commented:
"The system unfortunately becomes more complicated the more charges and more crime reports an officer needs to deal with."
I don't want to criticize QPRIME based on a single newspaper report, but the report hints at a much more general issue with complexity and business improvement.
This kind of complexity is neither produced nor resolved by SOA technology alone. The challenge for the business analyst is to understand the possible interconnections between multiple instances, to design services that render these interconnections properly, and to develop a workflow that allows users to access these services efficiently and effectively.
Tuesday, January 20, 2009
Security Drives Business Model?
In my previous post, Business Model Drives Security, I indicated that security requirements should follow from the business model - in the sense of the value proposition of the business.
In his latest post, In Person Credit Card Scam, Bruce Schneier reports a fraud involving a fake credit card and a trick telephone call, and remarks that "Anyone reading this blog would know enough not to call a number given to you by the potential purchaser, but presumably many store clerks don't have good security sense."
Some of the comments to Bruce's blog point to alternative solutions with varying levels of reliability - involving training, use of known telephone numbers, use of the number printed on the (fake) card - while some comments indicate related scams and vulnerabilities.
But perhaps the real answer lies further back. The basic retail process has evolved over centuries, from letters of credit and cheques to the modern credit card. Various security mechanisms have been added on, often intended more to protect the bank from the dishonest merchant than to protect the merchant from the dishonest customer. If you were going to design a secure system from scratch you wouldn't want to start from the current mess, but that's what we are faced with. So does SOA have a useful contribution here?
Clearly one of the key roles in this collaboration is communications, and the key vulnerability in this particular example is the fraudster's ability to direct the merchant's communication to an imposter bank. As it happens, Bruce Schneier's security monitoring company Counterpane is now part of British Telecom. So one obvious opportunity for Bruce's team is to design security services that BT can deliver into this ecosystem, both protecting honest merchants against known threats and detecting new threats.
In his latest post, In Person Credit Card Scam, Bruce Schneier reports a fraud involving a fake credit card and a trick telephone call, and remarks that "Anyone reading this blog would know enough not to call a number given to you by the potential purchaser, but presumably many store clerks don't have good security sense."
Some of the comments to Bruce's blog point to alternative solutions with varying levels of reliability - involving training, use of known telephone numbers, use of the number printed on the (fake) card - while some comments indicate related scams and vulnerabilities.
But perhaps the real answer lies further back. The basic retail process has evolved over centuries, from letters of credit and cheques to the modern credit card. Various security mechanisms have been added on, often intended more to protect the bank from the dishonest merchant than to protect the merchant from the dishonest customer. If you were going to design a secure system from scratch you wouldn't want to start from the current mess, but that's what we are faced with. So does SOA have a useful contribution here?
Clearly one of the key roles in this collaboration is communications, and the key vulnerability in this particular example is the fraudster's ability to direct the merchant's communication to an imposter bank. As it happens, Bruce Schneier's security monitoring company Counterpane is now part of British Telecom. So one obvious opportunity for Bruce's team is to design security services that BT can deliver into this ecosystem, both protecting honest merchants against known threats and detecting new threats.
Saturday, January 17, 2009
Business Model Drives Security
Gunnar Petersen (What's Your Business Model?) quotes Ian Grigg (What's Missing in Security: Business - see footnote).
Gunnar takes an asset-based approach to building such a business model - what are the assets and how do we protect them. Gunnar has always argued that security should be proportional to asset value. I think that's a reasonable starting point, as long as we take a broad view of what counts as an asset, and whose assets are included. In particular, companies should expect to accept liability for any loss or damage to customer assets or third-party assets. See my post Services Like Laundry, where I argue that the potential liability for a dry-cleaner is based on the likely cost of replacing any damaged item, rather than just the cost of the dry-cleaning service.
But the asset perspective is not the only way of thinking about business-driven security. Consider a large retailer, deciding what policies and mechanisms to adopt against shoplifting. This will partly be based on the expected value of the shoplifted items - so the small higher-priced items may have higher levels of protection. But this should be balanced against the business process or cashflow perspective. Is it better to lock goods into cabinets, lose fewer to shoplifters but maybe sell fewer as well? Or is it better to encourage shoppers to handle the goods, resulting in more sales as well as more theft? This is clearly a business judgement, which should not be solely based on the value of the goods. [And see my post on Shoplifting]
Overall, we need a business model that defines the sources of business value. This may include a broad concept of asset, but also broad concepts of capability and viability. Then I absolutely agree with Gunnar and Ian - the security model must be driven by the business model.
Note: Ian's blog Financial Cryptography has an security certificate that is not recognized by Firefox or Internet Explorer, for reasons he explains in the comments to Gunnar's blog. Visit it at your own risk.
Note: The RESG hosted a masterclass on Security Risk Analysis and Management in January 2003. See report with my comment in the RESG Newsletter (RQ28 Feb 2003).
See also: Services Like Laundry (April 2008), Event Processing Example - Shoplifting (September 2008), Two Kinds of Business Model.(December 2008), Security Drives Business Model (January 2009),
The bit that's missing is the business. Instead of asking "What's your threat model?" as the first question, it should be "What's your business model?" Security asks that last, and only partly, but asking questions like "what's are the risks?"Clearly this is what I'm calling Business Model B - what is the source of value for this enterprise - rather than Business Model C - the line-and-box diagrams drawn by Enterprise Architects. See Two Kinds of Business Model.
Gunnar takes an asset-based approach to building such a business model - what are the assets and how do we protect them. Gunnar has always argued that security should be proportional to asset value. I think that's a reasonable starting point, as long as we take a broad view of what counts as an asset, and whose assets are included. In particular, companies should expect to accept liability for any loss or damage to customer assets or third-party assets. See my post Services Like Laundry, where I argue that the potential liability for a dry-cleaner is based on the likely cost of replacing any damaged item, rather than just the cost of the dry-cleaning service.
But the asset perspective is not the only way of thinking about business-driven security. Consider a large retailer, deciding what policies and mechanisms to adopt against shoplifting. This will partly be based on the expected value of the shoplifted items - so the small higher-priced items may have higher levels of protection. But this should be balanced against the business process or cashflow perspective. Is it better to lock goods into cabinets, lose fewer to shoplifters but maybe sell fewer as well? Or is it better to encourage shoppers to handle the goods, resulting in more sales as well as more theft? This is clearly a business judgement, which should not be solely based on the value of the goods. [And see my post on Shoplifting]
Overall, we need a business model that defines the sources of business value. This may include a broad concept of asset, but also broad concepts of capability and viability. Then I absolutely agree with Gunnar and Ian - the security model must be driven by the business model.
Note: Ian's blog Financial Cryptography has an security certificate that is not recognized by Firefox or Internet Explorer, for reasons he explains in the comments to Gunnar's blog. Visit it at your own risk.
Note: The RESG hosted a masterclass on Security Risk Analysis and Management in January 2003. See report with my comment in the RESG Newsletter (RQ28 Feb 2003).
See also: Services Like Laundry (April 2008), Event Processing Example - Shoplifting (September 2008), Two Kinds of Business Model.(December 2008), Security Drives Business Model (January 2009),
Friday, January 09, 2009
Services Wake - A New Label for SOA?
Maybe the label "SOA" has gone for a Burton, but the concepts and principles of service-orientation remain important. There has been some good discussion on the CBDI Forum Linked-In group, prompted by the SOA obituary by Anne Thomas Manes.
If SOA is dead, long live services. Services (like the hero of the Irish song Finnigan's Wake) are not dead, they are only sleeping, and need to be woken up.
So do we need a new label? Some commentators have indicated that they would be happy to see the end of all such labels, but some have suggested a few alternatives.
I particularly like the last of these. (Perhaps some of my readers will guess why.) But I guess the CaaS label is the most likely to fly.
If SOA is dead, long live services. Services (like the hero of the Irish song Finnigan's Wake) are not dead, they are only sleeping, and need to be woken up.
So do we need a new label? Some commentators have indicated that they would be happy to see the end of all such labels, but some have suggested a few alternatives.
- BaaS - Business as a Service
- CaaS - Capability as a Service
- CBDI - Capability-Based Dynamic Intelligence
- ELVIS - Enterprise Layered Virtual Information Services
- TAFKAS - The Architecture Formerly Known as SOA
- VICO - Virtualized IT Capability Orchestration
I particularly like the last of these. (Perhaps some of my readers will guess why.) But I guess the CaaS label is the most likely to fly.
Thursday, January 08, 2009
Event Processing and Security
What is the relationship between Complex Event Processing (CEP) and Application Security (asks James McGovern)? My answer is that there is not one single relationship, but there may be relationships in both directions.
Perhaps the most obvious relationship is that event processing technologies can be used to support application security, helping to detect security threats and to monitor security compliance.
But there is also a reverse relationship. A CEP system may itself have vulnerabilities, and Application Security principles may be needed to address these vulnerabilities. For example, any event detection system is vulnerable to deceptive events, known as chaff.
Actually, there are two different concepts of chaff at play here. Some CEP experts use the term as it is used in agriculture, to refer to uninteresting or unwanted stuff. For example, see David Luckham and Mark Palmer, Separating Wheat from Chaff (RFID Journal, October 2004).
Meanwhile, security-conscious CEP experts, such as my friend Tim Bass, use the term as it is used in radar defence, to refer to decoy events. Chaff are pieces of metal designed to create misleading information for enemy radar systems. (Interestingly, in the second world war, each side invented this technique but hesitated to deploy it, for fear of revealing the technique to the other side.) For example, see Tim's post on Quintessential Event Processing (November 2008).
Both kinds of chaff increase the level of clutter, thus reducing the accuracy and reliability of event-driven systems. For example, see this article about the effect of clutter on security: Five ways to clean your firewall of clutter and stay secure.
David Luckham (again) mentions clutter as a user design issue (Dashboard Design) but even this kind of clutter has system consequences: it affects the effectiveness of a man-machine system; and this is not so very different from the consequences of (a different kind of) clutter on the effectiveness of a firewall.
From the accounts I have read of the Three Mile Island case, my understanding is that it was something akin to clutter (in David's sense) that made it almost impossible for the operators to respond intelligently and appropriately to a series of system alerts. The Three Mile Island case has important lessons for both event-processing systems and for security.
In terms of the manifold relationships between complex event processing and application security, which was where James' original question led us, I think that one of the possible relationships is that they both rest on the same set of fundamental concepts and theories - for example sociotechnical systems and cybernetics.
To understand these relationships more deeply, we need a general account of (secure event-driven) or (event-driven secure) systems, and this might involve more abstract but rigorous notions of chaff and clutter.
originally posted in the CEP discussion group on Linked-In
Perhaps the most obvious relationship is that event processing technologies can be used to support application security, helping to detect security threats and to monitor security compliance.
But there is also a reverse relationship. A CEP system may itself have vulnerabilities, and Application Security principles may be needed to address these vulnerabilities. For example, any event detection system is vulnerable to deceptive events, known as chaff.
Actually, there are two different concepts of chaff at play here. Some CEP experts use the term as it is used in agriculture, to refer to uninteresting or unwanted stuff. For example, see David Luckham and Mark Palmer, Separating Wheat from Chaff (RFID Journal, October 2004).
Meanwhile, security-conscious CEP experts, such as my friend Tim Bass, use the term as it is used in radar defence, to refer to decoy events. Chaff are pieces of metal designed to create misleading information for enemy radar systems. (Interestingly, in the second world war, each side invented this technique but hesitated to deploy it, for fear of revealing the technique to the other side.) For example, see Tim's post on Quintessential Event Processing (November 2008).
Both kinds of chaff increase the level of clutter, thus reducing the accuracy and reliability of event-driven systems. For example, see this article about the effect of clutter on security: Five ways to clean your firewall of clutter and stay secure.
David Luckham (again) mentions clutter as a user design issue (Dashboard Design) but even this kind of clutter has system consequences: it affects the effectiveness of a man-machine system; and this is not so very different from the consequences of (a different kind of) clutter on the effectiveness of a firewall.
From the accounts I have read of the Three Mile Island case, my understanding is that it was something akin to clutter (in David's sense) that made it almost impossible for the operators to respond intelligently and appropriately to a series of system alerts. The Three Mile Island case has important lessons for both event-processing systems and for security.
In terms of the manifold relationships between complex event processing and application security, which was where James' original question led us, I think that one of the possible relationships is that they both rest on the same set of fundamental concepts and theories - for example sociotechnical systems and cybernetics.
To understand these relationships more deeply, we need a general account of (secure event-driven) or (event-driven secure) systems, and this might involve more abstract but rigorous notions of chaff and clutter.
originally posted in the CEP discussion group on Linked-In
Tuesday, January 06, 2009
Has SOA gone for a Burton?
When British airline pilots disappeared during the Second World War, it was said that they had "gone for a Burton". Burton (then a popular brand of ale) became a cynical euphemism for being killed in action. [For further explanation of this phrase, see World Wide Words.] So is SOA missing in action?
Anne Thomas Manes of the Burton Group writes SOA is Dead, Long Live Services. I think what Anne is saying is not that SOA is dead but that "SOA" is dead - in other words the label "SOA" is out of fashion but the reality of SOA is still with us.
Anyway, Anne's tongue-in-cheek obituary of SOA (or "SOA") has provoked a lot of good comments. We can agree that SOA has to be done properly, and that probably means in conjunction with some other stuff. See my post on Technological Perfecta.
Does SOA need a new label? Or do we need a label for some larger group of stuff? Meanwhile, my friends in the SOA world can quietly continue applying the concepts and principles of SOA to practical business problems, unbothered by all this hype.
See also SOA Retrospective (June 2009)
Anne Thomas Manes of the Burton Group writes SOA is Dead, Long Live Services. I think what Anne is saying is not that SOA is dead but that "SOA" is dead - in other words the label "SOA" is out of fashion but the reality of SOA is still with us.
Anyway, Anne's tongue-in-cheek obituary of SOA (or "SOA") has provoked a lot of good comments. We can agree that SOA has to be done properly, and that probably means in conjunction with some other stuff. See my post on Technological Perfecta.
Does SOA need a new label? Or do we need a label for some larger group of stuff? Meanwhile, my friends in the SOA world can quietly continue applying the concepts and principles of SOA to practical business problems, unbothered by all this hype.
See also SOA Retrospective (June 2009)
Monday, January 05, 2009
From SOA to Better Judgement
In the comments to my post From Complex Events to Predictive Analytics, I have been debating with Hans Gilde about the causes of the credit crisis, and the possibility that better systems and technologies might have prevented the crisis.
Hans writes
Hans is not alone in making this kind of assertion, which appears to be largely based on an optimistic belief in the power of technology to solve problems. But my understanding of economics is that it is a pessimistic (or "dismal") science, in which periodic crises are pretty much unavoidable. Like forest fires or domestic arguments, they can be delayed or suppressed for a while, but that simply makes them bigger when they eventually break out. Certain politicians claimed that "we will never return to the old boom and bust" but, as we now discover, presided over a much larger boom and bust [Channel Four News]. The idea that there is any technocratic solution to fundamental economic problems seems like hubris.
I am also worried by Hans' appeal to "better judgement". Of course we'd all like to think that SOA produces better judgements, so I tried looking for evidence of this, but an internet search for "SOA" and "better judgement" mostly found pages in which the words "better judgement" were preceded by the words "against" or "despite". Oh dear.
But even if SOA does produce better judgements, there is still a problem: better for whom? The service-oriented world is essentially a distributed one, with no central judge, no supreme court. So there is no fixed standard of value.
There was an interesting example just yesterday on a BBC Radio programme called More or Less. Paul Wilmott (described by the BBC as a "financial mathematics guru") explained how a typical bonus system motivates financial traders to copy one another in order to maximize their expected bonus, even though this inhibits portfolio diversification and therefore reduces the overall stability of the bank. So what counts as a better judgement for the trader does not necessarily count as better for the bank and its customers. If you provide better information to the trader, this may well help the trader maximize his bonus, but doesn't necessarily produce better results for anyone else. (The argument can also be found on Paul's blog, in a post called Science in Finance V: Diversification.)
Of course it's easy enough to see what the problem is here - it is that the bonus system is wrong. But if you design computer system improvements without considering the inevitable imperfections in human systems, you are very likely to get a nasty shock.
In any case, even if you had a bonus system that perfectly aligned the motivation of the trader with the interests of the bank, and even if you could produce a computer system that perfectly calculated the True Value of every asset in the world, traders wouldn't use these calculations because they wouldn't generate enough money. What traders really want is a recursive system that calculates what every other trader thinks the value is, in order to spot market movements a few moments before everyone else. But why would anyone expect the widespread possession of such systems to make the global economy any less unstable?
There are many possible positive contributions that SOA might make to the operation and supervision of complex trading systems. But if we want these systems to produce good results for the right people, we have to do some really hard systems thinking, and not just hope that speeding things up is going to solve all the problems.
Hans writes
"if people had not deluded themselves into ignoring obvious risks, they would have put more realistic calculations in their spreadsheets, resulting in better judgment that would, AFAIK, have prevented the credit crisis"
Hans is not alone in making this kind of assertion, which appears to be largely based on an optimistic belief in the power of technology to solve problems. But my understanding of economics is that it is a pessimistic (or "dismal") science, in which periodic crises are pretty much unavoidable. Like forest fires or domestic arguments, they can be delayed or suppressed for a while, but that simply makes them bigger when they eventually break out. Certain politicians claimed that "we will never return to the old boom and bust" but, as we now discover, presided over a much larger boom and bust [Channel Four News]. The idea that there is any technocratic solution to fundamental economic problems seems like hubris.
I am also worried by Hans' appeal to "better judgement". Of course we'd all like to think that SOA produces better judgements, so I tried looking for evidence of this, but an internet search for "SOA" and "better judgement" mostly found pages in which the words "better judgement" were preceded by the words "against" or "despite". Oh dear.
But even if SOA does produce better judgements, there is still a problem: better for whom? The service-oriented world is essentially a distributed one, with no central judge, no supreme court. So there is no fixed standard of value.
There was an interesting example just yesterday on a BBC Radio programme called More or Less. Paul Wilmott (described by the BBC as a "financial mathematics guru") explained how a typical bonus system motivates financial traders to copy one another in order to maximize their expected bonus, even though this inhibits portfolio diversification and therefore reduces the overall stability of the bank. So what counts as a better judgement for the trader does not necessarily count as better for the bank and its customers. If you provide better information to the trader, this may well help the trader maximize his bonus, but doesn't necessarily produce better results for anyone else. (The argument can also be found on Paul's blog, in a post called Science in Finance V: Diversification.)
Of course it's easy enough to see what the problem is here - it is that the bonus system is wrong. But if you design computer system improvements without considering the inevitable imperfections in human systems, you are very likely to get a nasty shock.
In any case, even if you had a bonus system that perfectly aligned the motivation of the trader with the interests of the bank, and even if you could produce a computer system that perfectly calculated the True Value of every asset in the world, traders wouldn't use these calculations because they wouldn't generate enough money. What traders really want is a recursive system that calculates what every other trader thinks the value is, in order to spot market movements a few moments before everyone else. But why would anyone expect the widespread possession of such systems to make the global economy any less unstable?
There are many possible positive contributions that SOA might make to the operation and supervision of complex trading systems. But if we want these systems to produce good results for the right people, we have to do some really hard systems thinking, and not just hope that speeding things up is going to solve all the problems.
Sunday, January 04, 2009
Event Reduction
Opher Etzion on Event Reduction
An event processing system generally has some degree of event reduction and aggregation. There may be real-world events that are not represented as system events (for example, a continuous real-world process is reduced to a series of periodic instrument readings, and the intermediate points are not captured). And some events may be front-end captured but subject to some filtering so that not all of them are passed onto the business applications.
In both cases, the total behaviour of the enterprise system may be affected by the quantity and variety of events that get through to the business applications. Too many events, and the decision-makers may be unable to focus on the important ones. Too few events, and the enterprise may lack requisite variety. A good cybernetic systems design would include feedback loops, so that the quantity and variety of events can be adjusted until the enterprise works to the maximum efficiency and effectiveness.
So an event processing architecture needs some kind of variety regulator (“regulator” in the engineering sense, not the legal compliance sense) – some kind of monitoring and control. For example, changing the frequency of some measurement, or changing the filtering or aggregation rules.
In very simple event processing, the systems designers might possibly be clever enough to calculate the right measurement frequencies and event processing rules in advance, although I don’t know of any robust methodology for doing this. For complex event processing the mathematics start to get too difficult, and we need to rely on proper engineering and not just simple arithmetic.
“in some cases, the business benefit of event processing system is actually reducing the number of events, and focus the decision makers on the important events”
An event processing system generally has some degree of event reduction and aggregation. There may be real-world events that are not represented as system events (for example, a continuous real-world process is reduced to a series of periodic instrument readings, and the intermediate points are not captured). And some events may be front-end captured but subject to some filtering so that not all of them are passed onto the business applications.
In both cases, the total behaviour of the enterprise system may be affected by the quantity and variety of events that get through to the business applications. Too many events, and the decision-makers may be unable to focus on the important ones. Too few events, and the enterprise may lack requisite variety. A good cybernetic systems design would include feedback loops, so that the quantity and variety of events can be adjusted until the enterprise works to the maximum efficiency and effectiveness.
So an event processing architecture needs some kind of variety regulator (“regulator” in the engineering sense, not the legal compliance sense) – some kind of monitoring and control. For example, changing the frequency of some measurement, or changing the filtering or aggregation rules.
In very simple event processing, the systems designers might possibly be clever enough to calculate the right measurement frequencies and event processing rules in advance, although I don’t know of any robust methodology for doing this. For complex event processing the mathematics start to get too difficult, and we need to rely on proper engineering and not just simple arithmetic.
Market Predictions - CEP
What lies in store?
Complex Event Processing (CEP) guru David Luckham asks What are your predictions for 2009?What challenges?
Prediction is hard, especially for experts. What is easier is to identify some of the challenges that have to be addressed. For example:- Detection (Tim Bass)
- Interoperability (Opher Etzion, Tim Bass)
- Scope and Terminology (Opher Etzion, Paul Vincent)
CEP Products or Services?
David's starting point was a Forrester estimate of the market for CEP software and services. But what kind of services are these? Is Forrester just talking about conventional systems integration services - e.g. paying consultants to build and install your CEP systems? Or are we starting to see a genuine service economy based around the trading and collaborative processing of events?A certain amount of this kind of thing goes on in the security world, with specialist firms performing what is essentially collective event processing for a number of customers (Monitoring-as-a-Service). But apart from that, I haven't seen much evidence of a distributed economy of complex events.
But why might this kind of thing be particularly interesting in 2009? Because the prevailing economic environment may make it harder to justify people going it alone, building large and complex event-processing applications for their own use.
Shortening the lead
Most people are expecting 2009 to be a tough year. In such circumstances, there is a widespread reluctance to invest scarce resources on remote gains; any vendors trying to sell solutions to business will need to find innovative ways of shortening and tightening the lead between investment and return.For example, instead of vendors offering an elaborate set of CEP products, together with consultants skilled in wiring them together, there may be demand for out-of-the-box hosted solutions.
Shared Event Semantics
But in order to share events and event processing, firms will need to have a common event model. Which brings us back to some of the challenges identified by Opher, Paul and Tim ...And Not Just Complex Event Processing ...
These arguments don't just apply to complex event processing, but to many areas of new technology. I'll look at some other areas in subsequent posts ...
Subscribe to:
Posts (Atom)