Nearly finished my article on SOA testing.
Some people seem to think that SOA testing is all about testing web services – as if SOA testing is the same as JBOWS testing. But I am keen to emphasize other aspects of SOA testing as well.
One of the things I’ve been talking to a few people about is the testing of the service architecture itself. Some people have expressed doubts about this. Surely if there are any flaws in the architecture, these will become apparent when we test the other levels. Or perhaps we can just wait until the system goes live, and fix any architectural problems then.
I don’t agree. If we don’t test architecture, how do we know whether the architects have messed up? If we care about the quality of architecture, we need tests that are specifically designed to determine the correctness and robustness of the architecture, as well as tests to determine whether the intentions of the architects have been correctly realised. For example, we may decide to monitor calls between the layers, to verify that there aren’t any hidden dependencies, or to measure the performance overhead associated with each layer. Or when the architects have specifically designed multiple provision of some resource to avoid a single point of failure, we might monitor the utilization of this resource over a range of conditions to ensure that this aspect of the architecture is successful.
I suspect most people don’t bother – they assume that any flaws that don’t surface through other tests don’t matter. But I think this is an optimistic assumption, and system testing should never be based on optimism. Architectural testing may sometimes be difficult and expensive, and would therefore need to be justified against some assessment of business risk, but it should always be considered as an option.
So I’d welcome some comments from readers. Do you carry out tests that are specifically designed to verify and validate the architecture as a whole? If you don’t carry out such tests, is this (a) because you don’t have the time, (b) you don’t know how, or (c) you or your sponsors cannot see any value from such tests?