SOA Testing

Many articles, books, and conferences dealing with the SOA tend to ignore one of the most important topics in software development – testing. Often, testing is just an afterthought in many software development efforts. A lot has been said and written about this problem. However, it is important to note that testing in an SOA program plays a much more important and prominent role than in any other software development effort. Without a proper testing foundation, the whole SOA initiative will either fail or become too unwieldy or expensive to maintain.

One of the cornerstones of SOA is service reuse. Success of the SOA program is often measured through the amount of services created and reused. The biggest problem with testing in an SOA environment manifests itself when a service has several consumers and changes are made to it. How do you validate that this change does not impact service consumers? How do you determine the best way to deal with this change? Do you ask all of the service consumers to perform their own regression testing to make sure internal service changes do not impact them? Obviously, this is not an effective solution. With more and more services getting more and more reuse, you need a solution that minimizes the amount of manual testing you need to do but, at the same time, provides a clear understanding of how the service changes impact its consumers.

The services are composed of three primary elements – interface, contract, and implementation. Interface represents the protocol and communication mechanism between service and its consumers. Contract defines all of the interaction details such as message formats, SLAs, policies, etc. Service implementation is self-explanatory. A service can expose multiple interfaces and may potentially support multiple contracts. The key to understanding the impact on service consumers is to verify whether or not changes to any of the service elements invalidate how it behaves today. Changes that have no impact are called non-breaking; changes that modify the behavior are called breaking.

Each shared service needs to have an automated test created as part of its normal implementation. It will address two issues – provide an initial test bed for the service and automate all future testing needs. The test should inspect what changes are made to each service element. When service is modified in any way, the automated test suite should be executed to understand the impact of the changes. If all tests pass, the changes should be considered non-breaking and consumers should be unaffected. If any of the tests fail, this would indicate a breaking change and a new version of the service would need to be created. Alternatively, the impacted consumers can change but, ideally, breaking changes should trigger a new service version.

The biggest problem with SOA many companies face is lack of a consistent, comprehensive testing approach. Without automated regression testing for shared services, organizations are exposed to risk of high manual testing costs every time a service is changed or new consumer is added. Additionally, it can drive service versioning and serve as a formal validation mechanism that service consumers can trust. Automation can save millions of dollars in manual labor and ensure stability of the whole SOA platform.