Tuesday, November 17, 2009

Knowledge and Architecture

@EnterprisingA (Jon Ayre) suggests an #EAMantra "If you know it, add it to your architecture. If you think you know it, add that too."

My immediate objection to this rule was that it seems to turn the architecture into a kind of brain dump. A good architect knows many things, and if all these items of knowledge go indiscriminately into the architecture, the architecture gets rather cluttered.

@EnterprisingA 's first defence was to say that "the quality of a brain dump depends on the quality of the brain from which the dump comes". And of course only the architect gets to deposit knowledge into the architecture (otherwise there would be a free-for-all).

Unfortunately, the brilliance of the architect is no guarantee that the architect's thoughts all satisfy some meaningful criterion of quality. Intelligence doesn't necessarily mean always having better-quality knowledge, but it should mean having a better capability for processing and developing and filtering and revising knowledge. And as Dumbledore says, "Being - forgive me - rather cleverer than most men, my mistakes tend to be correspondingly huger".

So the architect's knowledge may well get into the architecture eventually, but it doesn't need to go straight in. But @EnterprisingA is (quite reasonably) worried about not writing stuff down. "Keeping it in your head until you are absolutely certain is a good way of never being certain (or being challenged)." "I'd like you to review the arch but it's not certain til it's reviewed so I haven't drawn it so you can't review it..." Of course that's true, but I don't agree that there are only two places to keep knowledge - either in your head or in the architecture. @rgechristie suggests a third possibility - an interim area for in progress work before it hits "the architecture" (that still needs to be accessible by all developers and consumers of the architecture).

***

My second objection was that the architect has a lot of knowledge that doesn't really belong in the architecture at all. There is a strong temptation to put too much into an architecture, and the architectural documents can easily get bloated with miscellaneous material that the architects imagine might be useful to developers and others. (Methodology documents are subject to the same tendency.) For example, the architect may review a design document, may spot a particularly egregious design flaw, and decide to add a rule into the architecture, so that the architecture evolves into a general-purpose design handbook. And if every pearl of wisdom spoken by the architect has to be preserved in "The Architecture", you end up with "The Baroque" - a highly decorated and convoluted style.

It is of course possible to slim down an architecture with too much detail, but in practice this doesn't happen as often as it should. It is much easier to add material to a document than to subtract it, so over time the documents get longer and harder to read.

***

Following our debate on Twitter, @EnterprisingA reformulates his mantra to specify architectural decisions only. "If you know it is the right architectural choice, add it to your architecture. If you think you know it is, add that too." Surely I can't disagree with the proposition that "architectural decisions should be in the architecture", can I?

Actually I do. Sometimes the best architectural decision is to leave something out of the architecture, to leave something deliberately open and unspecified, underdetermined rather than overdetermined. This is a principle of just-enough architecture, or Zen architecture. Sometimes less is more.

4 comments:

Jon H Ayre said...

In response and for clarity:

Richard provides a reasonable summary of the discussion and finishes, I believe in a reasonable position of accord between our viewpoints (although I am not necessarily fully in agreement with the final conclusion).

To clear up a few misunderstandings arising from the twitter 140 char limit:

Firstly, the original mantra was kept short and snappy, and this led to an initial misunderstanding, which I believe drove much of the remaining conversation. My retweeting of the mantra was not a correction as such, but a clarification of original meaning for Richard's benefit.

I do not at any level propose that an architecture is a dumping ground for general knowledge. What I do believe, however, is that a great problem in architecture is that many architects are unwilling to include something in their architecture that they are not 100% confident in for fear of criticism. One of the basic premises of architecture is that we must be able to work with uncertainty and provide an increased confidence in the direction of travel. Waiting for 100% certainty will result in an empty architecture. It is this issue on which the mantra focuses.

My comment on the quality of brain dump being dependent on the quality of brain, was of course a tongue in cheek comment (and included a smiley at the end to indicate this).

Richard's post then continues to focus on knowledge, but my mantra focusses on the architects knowledge that an architectural decision is correct (with a reasonable level of confidence). The architecture is therefore a repository for decisions, not knowledge, and these decisions are recorded as architectural answers in the form of a big picture that draws everything together into a coherent whole. For me, the discipline that focuses on the collection and recording of knowledge is analysis, not architecture.

I therefore agree with Richard's second objection "the architect has a lot of knowledge that doesn't really belong in the architecture at all", but this is not an objection to my mantra as already stated. I also disagree with Richard's assumption that addition of knowledge relating to what is architecturally right will increase the size of the architecture. For example, knowing that it is architecturally correct to simplify the business by doing eight apparantly different things in one single generic way will drastically reduce the size of the architecture, not increase it.

One thing I can agree with whole-heartedly in Richard's article is this "For example, the architect may review a design document, may spot a particularly egregious design flaw, and decide to add a rule into the architecture, so that the architecture evolves into a general-purpose design handbook." The architecture is NOT a design handbook. It is the first stage of design focussing on the whole picture rather than the point solution.

Finally, Richard makes the statement "Sometimes the best architectural decision is to leave something out of the architecture, to leave something deliberately open and unspecified". Now this gives me serious concern. Certainly (as the mantra suggests) if you do not know something is correct architecturally it is not correct to add it, but if you think it is correct (with a reasonable level of confidence) you absolutely should add it, if for no other reason than to allow it to be tested and challenged. The alternative is to withold your input and leave it up to others to develop an answer, only to turn round afterwards and say "I already new that" (and seriously annoy the designer who has just wasted valuable time re-investigating it. An architecture is not a crime drama, where the witholding of a subtle piece of information will improve the surprise at the conclusion by not giving away too much too soon.

Regards
The Enterprising Architect

Richard Veryard said...

Thanks Jon. There are several different strands here, which are worth teasing apart.

1. I agree with your general principle that you shouldn't wait for 100% certainty. However, this creates a governance issue. Are developers and project managers expected to use the contents of the architecture as if they were 100% certain, with some mechanism for correction and tidying up when things change? Or is there a confidence level associated with each part of the architecture, and developers and project managers are expected to take this confidence level into consideration when using the architecture? Does the architecture need to include the reasoning and evidence supporting each architectural decision?

2. Many people regard design patterns and styles as part of an architecture. There is lots of admiration in the software literature for architects like Frank Lloyd Wright who didn't just design the building but also designed the wallpaper. So I don't think there is a clear consensus about the boundary between architecture and analysis/design.

3. Jon's detailed explanation indicates a difference between knowledge and decision, and I am happy to regard the architecture as a collection of decisions supported by knowledge, rather than merely a collection of knowledge.

4. Jon argues that "simplifying the business by doing eight apparently different things in one single generic way will drastically reduce the size of the architecture". I agree that this is sometimes possible, but I also think that a single-minded quest to standardize things can often make an architecture much too complicated. For example, an architect might get it into his head that an international company should have a single global standard for names and addresses, and so try to impose a universal name-and-address schema across the enterprise.

5. Finally, leaving things out of the architecture isn't a question of withholding knowledge for other people to rediscover, because we have agreed that the architecture isn't a container of knowledge but a container of decisions. I believe in the principle of subsidiarity, which means that there is a proper balance between central decision-making and local decision-making. There is also a principle of neutrality, which means that the architecture should not be tied to any particular solution or process or technology. The architect may know that the corporation has already committed itself to SAP, but may decide (for various reasons) that this knowledge should not be coded into the architecture.

6. The temptation to overspecify is common in many fields of architecture. Old-fashioned houses used to have bedrooms and bathrooms and corridors. Modern houses have Master Bedrooms with Ensuite Bathrooms, because the architect in his wisdom has already decided which bedroom the parents SHOULD occupy. I hate this kind of presumption, and I think it is inappropriate in enterprise architecture as well.

Jon H Ayre said...

Frustratingly, I now find that we have reached a genuine consensus of opinion, and I can find very little to disagree with you on in that last summing up. Ho hum...

...Apart from one last thing (in true Columbo style):

You ask "Are developers and project managers expected to use the contents of the architecture as if they were 100% certain, with some mechanism for correction and tidying up when things change?". No.

The true skill of architecture is to create the target future architecture with enough forward thought to allow projects to deliver steps towards that target. Thus, when the end point moves by small or relatively small amounts, the direction of travel can be altered by the next projects without redirecting the current ones. I am not suggesting that this can always be achieved, but it is a strong technique for ensuring that changes to the future architecture can be absorbed by the time gap between the end point and the current position of the in flight projects.

Regards
The Enterprising Architect

Richard Veryard said...

There will always be uncertainty and error. Good architecture may strive to reduce this and to reduce its impact. Then it is for Governance to decide where (by whom) the ensuing complexity is handled, and who pays for any future change. Obviously this is a particularly important issue when multiple organizations are involved (outsourcing).