Wednesday, October 24, 2012

Evolving the Enterprise Architecture Body of Knowledge

#entarch #EABOK There is a considerable "body of knowledge" in the enterprise architecture world. Among other things, this includes

  • Founding documents that everyone references, such as Zachman's 1987 paper for the IBM Systems Journal, and the MIT book on Enterprise Architecture as Strategy.
  • Some standards, such as ISO/IEC 42010 and RM/ODP.
  • Various frameworks that package this knowledge into some kind of semi-formal framework, such as TOGAF and DODAF/MODAF, as well as the Zachman framework. The Federal EA framework is incorporated into US law via the Clinger-Cohen Act.
  • Various tools that encapsulate portions of this knowledge.
  • Some attempts to formalize and ground this knowledge. There is some interesting work in the defence community to produce an EA ontology (MODEM) to replace the existing MODAF/DODAF metamodel. Meanwhile, there is a growing academic literature, much of it from the Netherlands for some reason.
  • Huge amounts of commentary and proposed additions/revision by individual authors and bloggers. There is also a considerable admixture of insight and obfuscation from the large consultancies and analyst firms.

The body of knowledge as a whole can be understood as a set of definitions ("this is what we shall call a business function"), assertions and observations ("loosely coupled structures are more flexible"), instructions/injunctions ("always agree your principles before planning your systems"), and examples (mostly artificial or anecdotal), together with a bunch of rather spurious or irrelevant claims ("this is a mathematically proven technique", "this classification was invented by the ancient Greeks", "this is what everybody means by 'complexity' "). The body of knowledge also relies significantly on some terms that usually remain undefined, such as "alignment".

This body of knowledge as a whole has been subject to a number of criticisms.

  • It is incomplete, inconsistent, imprecise, muddled, simplistic.
  • It is impractical, fails to deliver value, not fit for purpose (however that purpose may be understood).
  • It carries a hidden ideological agenda (which conveniently suits the commercial interests of the large software and IT services companies).
  • It lacks a proper base of empirical evidence - much of the knowledge is received wisdom and rehashed fragments from other sources.
  • It relies on "subject matter experts", who often have no experience or training in knowledge research and are merely trading their own preconceptions.

In response to these perceived flaws in the collective body of knowledge, many EA practitioners espouse a personal body of knowledge, which is smaller and hopefully more consistent than the conventional/collective body of knowledge. This personal body of knowledge is often presented explicitly as an alternative to the conventional body of knowledge, for example rants against TOGAF or Zachman. However, personal bodies of knowledge are not immune from the same criticisms, and often suffer from methodological syncretism.



How has the collective body of knowledge been developed and validated? Typically a combination of the following
  • This is what everybody knows.
  • This is what every good architect knows.
  • Here's an interesting new idea, so let's bung it in somewhere.
  • Our customers like it, so it must be right.
  • This bit of TOGAF is obviously rubbish, so let's chuck it away and put something else in its place.


Imre Lakatos said that a research programme can be progressive or degenerative. A progressive research programme is one that enhances the explanatory or predictive power of a body of knowledge. (For example, the development of new medical knowledge is progressive if and only if it clearly contributes to more effective healthcare.) A degenerative research programme is one that merely adjusts the body of knowledge to explain away inconvenient results. (Hubert Dreyfus has argued that AI was a degenerative research programme, because it ran into unexpected problems it could not solve.) I have frequently observed that much of EA looks more like mediaeval scholasticism (taxonomy for the sake of taxonomy) than modern science.

I know that some of the readers of this blog aspire to push forward the EA body of knowledge in various ways, including the next version of TOGAF and the replacement EABOK - so my challenge to you guys is this: How are you making sure that your work is progressive and evidence-based?

15 comments:

Nick Malik said...

Hi Richard,

I responded to your tweet with a post of my own.

http://eabok.org/node/52

--- Nick

Richard Veryard said...

Reading Nick Malik's thoughtful response on the EABOK-2 website Is the EABOK progressive and evidence based?, I accept that I may have jumped to conclusions about the EABOK-2 goals.



There are three possible goals for the EABOK-2 initiative. The first is to create new EA knowledge. The second is to "curate" EA knowledge - in other words, to create a collection of the best bits of existing knowledge. And the third is simply to create a guide to what exists, without attempting to evaluate it.


I should also explain further what I mean by "body of knowledge". Nick says the medical profession does NOT have a single body of knowledge, but I think it makes sense to talk about the collective knowledge of the medical profession, perhaps even including alternative practices, as an extremely large body of knowledge, which may well contain all sorts of conflicting ideas. I take the term "Guide to THE body of knowledge" to imply both (a) that there is a single sprawling body of knowledge contained in any number of documents as well as practitioners' heads, and (b) that EABOK-1 and EABOK-2 are two different guides to this body of knowledge. If I look on Amazon I can find many guides to China, but presumably there is only one China.


Now, if I'm writing a guide to China, I may attempt to write a purely factual guide, with no subjective judgement. For example, I may decide to include every city above a certain size. Or I may wish to write a subjective guide, in which I try to select the most interesting or beautiful cities. I don't think the authors of EABOK-2 have got this choice - the selection is inevitably going to be based on some collective editorial judgement about the validity and importance of the ideas included.


And there is another difference between writing a guide to China (which probably doesn't significantly affect the reality of China) and writing a guide to a body of knowledge, which if successful will become incorporated into that body of knowledge.


Any guide needs to be helpful and accurate, but I think a guide to a body of knowledge should also contribute to improving the body of knowledge as a whole. I am having second thoughts about whether this counts as research, and therefore whether it is fair to apply Lakatos's notion of a progressive research programme to the EABOK-2 initiative itself. It may be more more appropriate for the EABOK-2 authors to use Lakatos's criterion when deciding which bits of EA research to include in the guide. This would send an important message to EA researchers (in the same way that curators of art exhibitions inevitably influence the work of living artists) and would therefore influence the future development of EA knowledge.


Nick asserts a principle that each contribution to the guide should be progressive and evidence-based, but of course this is not the same as saying that the guide as a whole will be progressive and evidence-based. As every architect should know (ironic smile), the qualities of the whole are not simply a function of the qualities of the parts.

Ric Phillips said...

EABOK-2's success will depend on attracting a broad community of contributors, and within that a strong sub-set of people whose intellectual bias is, as you say, fact based and progressive.

Medicine is an interesting and perhaps apt analogue because while we may assume the clinical efficacy of medice is grounded in the rigorous application of basic medical science, in fact, many advances in medicine have been made through clinical praxsis.

The simple fact of the matter in our profession is there is currently no citadel. There are no established organs of professional quality and practice that provide for a framework of peer review and a progressive collegial discourse.

The field is awash with hero-philosophers who far from providing tempered and tested solutions, are more concerned with colonising and defining the problem domain. To which, of course, their approach is the most rational answer. And I confess to falling prey to that aspect of my own temperament.

In the not too distant past - less than half a human lifetime - it was clear that universities were the institutions that provided the body-of-knowledge function. In concert with professional, government, and industry bodies of course, but universities provided the central coordinating and legitimising function.

I am the first to acknowledge we are moving into the post-University age as the internet has opened up radically different ways for society to create, collated and codify knowledge.

But while this is the case, the higher education sector has, and will have a crucial role to play in the establishment of genuine 'bodies of knowledge" that are tested by a body of qualified peers and backed up by objective, open and publicly challengeable, evidence based research. For a start they have money, and a sustained infrastructure specifically set aside for such activity. And while far from perfect, they have generations of experience in proving knowledge.

I am keen to contribute constructively to any attempt to codify, improve and share the knowledge of how enterprise architecture is done. But I must confess, this blind spot within the profession eludes me.

I cannot see why initiatives like the EABOK-2 do not consider the involvement of higher education in such endeavours to be mandatory.

Richard Veryard said...

Thanks Ric

For much of the history of medicine, innovation depended on brilliant individuals taking huge risks with their patients' lives. Medicine has now developed some fairly strict knowledge protocols, so everyone can perceive the difference between a promising idea and a proven approach. Whereas in enterprise architecture, the word "proven" generally means "I talked to someone at a conference who told me it has worked at least once".

Meanwhile, the universities and business schools have a tendency to produce a lot of fragmented and often intellectually shallow work. (Any academics reading this are welcome to regard themselves as worthy exceptions to this trend.) Much of the serious research in systems architecture and related topics has taken place outside or at arm's length from universities, in R&D labs funded by the computer and telecoms companies, and by the US government.

Nonetheless, involving higher education in initiatives such as EABOK-2 ought to help the whole community (including the academics) to become more joined-up, more practically grounded and of course evidence-based, and I agree that such involvement is to be encouraged.

Adrian Grigoriu said...

The EA BoK should "curate" the existing knowledge rather than create it (EABoK should not be yet another player) or solely collect existing bits without intelligent processing, since it will have to include then multiple definitions and interpretations of the same concepts.

A body of knowledge does not necessarily exist in a single place, even though compendiums or sites can summarise the BoK, more or less.

It is the agreement on the BoK that matters.

And BoK evolves in time with bits added, changed, updated and integrated from many other works.
One can only summarise the bits that people agree upon at some stage.

But for now, there is little consensus on answers to the key EA questions like what it is, what its scope and purpose are, what does it deliver... and what is the relation to strategy.

There is no understanding of why are so many different approaches, frameworks, certifications, books that have little in common.
It is hard to believe that a body of knowledge that does not consider that would be adopted or representative.

One way to look at it is to define what EA delivers first and hopefully, the rest of the body, would fall in place.

Work for a body of knowledge should be performed by a rather independent body, possible internationally distributed, without no reasons for bias, which body should study the existing situation first, argue a solution from a few choices, and propose and integration of the best of breed of the existing approaches.
A governance board should make the hard decisions.

I thought that this should be the kind of work an organization like Open Group would do rather than obstinating to "create" its own original method, producing in fact its own body of knowledge.

Richard Veryard said...

Excellent comment, Adrian.

What then does this imply for the responsibility of the curator? Is the curator responsible for the quality of the knowledge curated? If the knowledge is flawed - incomplete or inconsistent or lacking evidence - is the curator trying to cover up these flaws, to make the knowledge appear more complete and consistent and valuable than it really is?

On the contrary, I believe it is right to expose any potential flaws, in order to encourage researchers and practitioners to develop improved knowledge and improved practices. This is an essential characteristics of a progressive programme.

Adrian Grigoriu said...

As with other roles, the curator the EA BoK team) should have the responsibility invested by its mission. For instance, analysis of existing approaches, alternatives formulation, public consultation, integration of outcomes... But no more.

The key decisions should be made by a different body, the governance Board simply because, it is hard to do the work and then make decisions on the same issues, without bias or perhaps being accused of it. This could be a very sensitive process since some popular approaches will suffer.

In the process, every method owner should be given the chance to expose its framework mechanism.

The curator's mission should indeed be to eliminate "flaws" like inconsistencies, narrow points of view, biased definitions etc. Otherwise, what would be the point of the work?
For instance, a new definition for the EA may be necessary by pruning out and selecting the good bits of the many existing ones.

Still, this could be a painful process that would otherwise happen naturally. True this would happen during much longer period of time while egos and interests fade away.

Richard Veryard said...

Expose flaws or eliminate them? Sometimes one has to expose flaws in the short term, hoping that this will lead to work that eliminates the flaws in the longer term.

When I am using a body of knowledge, I prefer to be made aware of its limitations, rather than seduced into an illusion of perfect consistency and completeness.

Compare with medical knowledge. Sometimes there is conflicting evidence about alternative treatments - one study seems to show that surgery is more effective than drugs, while another study seems to show the opposite. The doctor makes a professional judgement for a specific patient, based on an evaluation of the available evidence.

And if the curators of a body of knowledge try to maintain a strict bureaucratic quality threshold, this introduces something called Publication Bias, which (among other things) makes it harder for novel and experimental approaches to be considered, and acts as a brake on innovation.

So I would be cautious about trying to present any body of knowledge as being beyond doubt. In my opinion, a body of knowledge should support professional judgement, not replace it.

Adrian Grigoriu said...

A BoK may consist of dilemmas but not in the definition of the discipline because we might be talking then about different concepts.

Right now the EA BoK consists of too many definitions in terms of what it is, what is it for, what are its boundaries...

To shorten the BoK process and move things forward, rather than listing all definitions and alternatives, an independent team could propose a choice out of the many existing ones.
But true only time can tell.

Richard Veryard said...

Good point, Adrian

However, definitions are of no value in themselves: they may provide information, they may serve as a necessary precursor to knowledge, but should not be counted as knowledge. A collection that only contained definitions and categories and notations would be (at best) an Embryo of Knowledge, not a mature Body.

Perhaps a long time ago, doctors argued that it wasn't proper medicine unless you cut people up and drained their blood. Enterprise architecture is still at the stage where people are arguing that it's not proper EA unless you cover the patient with leeches. (Okay, I'm being sarcastic, but you know what I mean.)

In which case, we would need to make sure that the leeches don't have editorial control over the Body of Knowledge.

(No reference intended to any real person or organization. Obviously.)

Adrian Grigoriu said...

Richard,

definitions ultimately establish the subject and scope of the BoK.
They help us narrow down the knowledge that belongs to the same BoK.

With numerous, conflicting or partial definitions the whole BoK may go astray.

For instance now, we are talking about EA as "the glue between strategy....", EA as business model, EA as operating model, EA as IT, EA as software architecture ...

Otherwise, it looks to me that a few organizations might be already trying to hijack the EA, pretending to serve the community, by establishing certifications based on no body of knowledge at all.

On the other hand, there are individuals (like myself) and organizations that produced their own definitions and method, proposing, more or less, their own body of knowledge.

This existing knowledge must be rationalised if we are trying to shorten the BoK process.

What is most important, but missing today, is critical thinking. That slows down the natural BoK formation.
People rather than bothering to arrive at an own opinion, adopt blindly points of view only because they were emitted by personalities.

James Lapalme said...

With regards to your three possibilities, it believe the EABoK must follow a fourth path.

I believe that the success of the EABoK-2 initiative will lie in the Consortium's capacity to identify quality knowledge in the communities of academia and practitioners and transform that knowledge into ready-to-use knowledge. However, the Consortium must establish bridges between academia, practitioners and organizations in order to :

- facilitate the flow of knowledge
- Influence research to target knowledge areas that are shallow
- Promote quality ready-to-use knowledge

In summary, this is not your first option because it is not about knowledge creation, but rather knowledge transformation. This is not your second option because the Consortium should play an active role : knowledge transformation and influencing research efforts. This is not your third option, because the Consortium should strive to promote quality knowledge that is scientific from the stances of positivism (Empirically Tested), pragmatism (Design science or action-research) or constructivist (Qualitative research). If it promotes knowledge that is not scientific, the Consortium should be up front about it... something like pending verification.

calfred56 said...

Thanks to Richard for inviting me to post here. First, a couple of thoughts on using medical BOK as a model.

Similarities:

The medical BOK has some usefulness:

There is a valid whole-part comparison: The relationship of ET to EITA has similarities to the relationship of Medicine to one of its major constituencies, such as Cardiology.

> Cardio health is as vital to a person's life as EITA is to EA.

> But individual health is much more than cardiology

Sometimes what's good for individual health might tradeoff vs. cardio health.

Cardiologist need to contribute to BOK but shoudn't dominate as their perspective is generally narrower than the top-level.

Differences:

Health care is a system, but not an enterprise per se

Components (hospitals, clinics, pharmaceutical companies, imaging companies, HIS vendors) are a much more loosely coupled ecosystem than a typical enterprise.

calfred56 said...


This is a comment on the question of whether there is one common BOK for EA and EITA or separate.

My thought is that a compromise between these two approaches is needed.

There is enough commonality between the two disciplines that it would be a shame to duplicate.

On the other hand, there are significant differences (more about this shortly) that combining them is asking for problems.

I don't know what the logistics are, so I'm not going to recommend a specific solution. But it seems like it would be a good idea to manage the common parts in one place and include them in separate BOKs.

Rationale:

EA and EITA have different, but overlapping focus
> Both enterprises and IT are intentional systems

1. Generally, IT is subordinate to and enabling resource for an enterprise

2. "Enabling resource" implies a capability that is subject to certain constraints, and shouldn't just have requirements imposed on it.

3. Enabling resources typically have lead time to develop and deploy a capability, and need to be both ahead of its "customer", but also somehow aligned with the customer's needs. In practice, this can be very hard.

4. EA and EITA architectures must cooperate and collaborate.


The BOK(s) must support and be consistent with the cooperative missions.

Systems and architecture are a fundamental concern they share. The part of the BOK ought to be part of the common part:

The separate aspects aren't entirely divorced, but are differentiated by priority and focus.

For example:

EITA must be focus on concerns of scalability, information security, privacy (esp in health care), availability, data retention, and others. These concerns are also important to EA, but they don't elevate to top-of-mind.

By contrast, EA has no choice but to be concerned with sales trends, geographic markets, competitors, distribution channels, supply chains, regulatory trends, etc. These concerns are important for EA because they frame its context, but they fall largely off the map until they affect an existing system or require a new one.

Richard Veryard said...

Obviously there are both similarities and differences between EA and EITA. Charlie thinks that the differences outweigh the similarities, and therefore advocates two separate practices (disciplines) with separate bodies of knowledge.

The reason I am not keen on this separation is that it seems to institutionalize the gulf between two different specializations, and reduces the likelihood of common ground (shared concepts, shared techniques, shared tools) between EA and EITA.

I think it makes more sense to talk about a single Body of Knowledge with a number of specialist extensions for different purposes and contexts. Clearly EA and EITA represent different purposes and contexts, but that doesn't necessarily mean they must be constituted as separate disciplines. Surely we would not want the EITA community and the EA community to go off in completely different directions?

Let us remember that there are many different specialisms within medicine. So in addition to the general body of medical knowledge that all medical practitioners are expected to learn, there will be extensions for each specialism. But a common basis must remain, and there is an important role for generalists ("general practitioners") as well as specialists.

EA and EITA may indeed be quite different, but we should have more people who can do both.