Project-Oriented SOA

Read my article on the Project-Oriented SOA in the SOA Magazine.

http://www.soamag.com/I21/0808-2.asp

SOA Testing

Many articles, books, and conferences dealing with the SOA tend to ignore one of the most important topics in software development – testing. Often, testing is just an afterthought in many software development efforts. A lot has been said and written about this problem. However, it is important to note that testing in an SOA program plays a much more important and prominent role than in any other software development effort. Without a proper testing foundation, the whole SOA initiative will either fail or become too unwieldy or expensive to maintain.

One of the cornerstones of SOA is service reuse. Success of the SOA program is often measured through the amount of services created and reused. The biggest problem with testing in an SOA environment manifests itself when a service has several consumers and changes are made to it. How do you validate that this change does not impact service consumers? How do you determine the best way to deal with this change? Do you ask all of the service consumers to perform their own regression testing to make sure internal service changes do not impact them? Obviously, this is not an effective solution. With more and more services getting more and more reuse, you need a solution that minimizes the amount of manual testing you need to do but, at the same time, provides a clear understanding of how the service changes impact its consumers.

The services are composed of three primary elements – interface, contract, and implementation. Interface represents the protocol and communication mechanism between service and its consumers. Contract defines all of the interaction details such as message formats, SLAs, policies, etc. Service implementation is self-explanatory. A service can expose multiple interfaces and may potentially support multiple contracts. The key to understanding the impact on service consumers is to verify whether or not changes to any of the service elements invalidate how it behaves today. Changes that have no impact are called non-breaking; changes that modify the behavior are called breaking.

Each shared service needs to have an automated test created as part of its normal implementation. It will address two issues – provide an initial test bed for the service and automate all future testing needs. The test should inspect what changes are made to each service element. When service is modified in any way, the automated test suite should be executed to understand the impact of the changes. If all tests pass, the changes should be considered non-breaking and consumers should be unaffected. If any of the tests fail, this would indicate a breaking change and a new version of the service would need to be created. Alternatively, the impacted consumers can change but, ideally, breaking changes should trigger a new service version.

The biggest problem with SOA many companies face is lack of a consistent, comprehensive testing approach. Without automated regression testing for shared services, organizations are exposed to risk of high manual testing costs every time a service is changed or new consumer is added. Additionally, it can drive service versioning and serve as a formal validation mechanism that service consumers can trust. Automation can save millions of dollars in manual labor and ensure stability of the whole SOA platform.

Context Delivery Architecture

At the last Gartner Application Architecture, Development, and Integration (AADI) Summit held in Orlando, FL at the beginning of June, William Clark introduced the concept of Context Delivery Architecture (CoDA). It is an architecture that is aware on the end user’s context such as location, preferences, identity, etc. and delivers the information that is most suited for it. Another way to describe it is WYNIWYG (what you need is what you get) services. The basic idea is that users’ specific context will drive what information they receive, how the applications interact with them, and where the processes take them.

Despite Gartner labeling CoDA as an emerging trend, I don’t believe it is a new concept. Context and location aware applications have been in existence for a while. Think back to the Internet bubble days when all kinds of schemes were designed to deliver coupons, advertisements, and other “useful” information to your mobile devices when you got close to certain location. RFID and its applications became the staple of context aware applications. Even Gartner based its research on these trends. It is true, however, that CoDA has not yet become mainstream and is moving up the Gartner hype cycle curve.

CoDA is still very immature. The vision is that CoDA applications will ubiquitously run on a variety of devices, technologies, and platforms. For this to become a reality, technology needs to be created that would allow the same services to be delivered to a variety of platforms that possess the same context aware capabilities. Users should benefit from being mobile, not be hampered by it. For example, salespeople that left the office for the client visit should be able to obtain specific customer information, find out sales status, and view the whole relationship picture immediately on their preferred device. The same capabilities should exist on all mobile platforms, which will truly make context aware applications possible. At the same time, mobile devices should evolve to ubiquitously interact with the network. Whether a WiFi, cellular, or any other kind of network is available should not prevent the application and the device from performing their functions.

Even though CoDA was billed by Gartner as the next step in the evolution of SOA, I don’t think it fits into the same paradigm. SOA’s primary goal is to create composite applications through the leverage of existing services. EDA, or as Gartner likes to call it, Advanced SOA, pursues the same objective, except that instead of services, the same events are sought to be consumed. By contrast, CoDA aims to enhance user’s experience through the knowledge of his/her context and tailor the application behavior to it. While it builds on the concept of reusable services that would deliver the right information at the right time, the whole concept has nothing else in common with SOA. In my opinion, CoDA is a move towards more intelligent applications but it is definitely not the next evolution of SOA.

CoDA still has a long way to go. It is an exciting concept that has science fiction written all over it. However, the technology, devices, networks, and people are nearing the point when context aware applications will become commonplace. The exciting thing is that I don’t think we have much longer to wait.

Starting on Your SOA Journey

Many industry leaders believe that SOA should be started small and evolve over time. The argument is that this approach gives companies an opportunity to implement SOA practices on a small scale, test them out in a controlled environment, and understand how everything would work within the organizational boundaries. This is not a bad idea. However, I believe that the most effective way to introduce SOA is to build out the whole infrastructure, introduce the necessary technology, and establish all the patterns, best practices, reference architectures, and governance mechanisms before creating a single service.

The reasoning for this is simple. SOA on a small scale is not SOA. It is just a bunch of services. SOA’s goal is to create and leverage services across the organization. A single project or a couple of services cannot achieve this. Furthermore, effective governance, best practices, and lifecycle processes cannot be established on a small scale. They need to be designed and implemented with the large scale in mind. Testing them on a single project is not only impractical – it doesn’t provide any knowledge of how SOA will truly work within the organization.

Any successful SOA implementation will eventually have all of its elements in place – infrastructure, technology, governance, practices, processes, and people. Consider the impact of growing all this organically. You will end up with a hodge-podge of services implemented on different platforms using different approaches and design patterns. The technology set will be inconsistent. Governance mechanisms that typically tend to be established late in the game will most likely allow inadequately designed and implemented services to go into production. All this would have to be remediated at some point of time. Imagine the effort required to clean up years of organic growth! Most companies simply move on and leave the mess behind.

Now imagine what will happen if all of the SOA elements are in place from the very beginning. No rework, re-platforming, or cleanup will be required. All of the services will reside on the right platform that can be scaled for future demands, all of the best practices will be followed, and the governance mechanisms will be able to catch most, if not all the subpar services. The company will be able to reap SOA benefits right away without having to do the costly cleanup or conversion.

Of course, waiting to complete all the preliminary work can take years. No company, regardless of how strong its commitment to SOA is, can wait that long to start seeing the benefits of something that will require a lot of upfront investment. Thus, the most pragmatic approach is to introduce as many SOA elements as possible that will provide the most complete and consistent SOA foundation for the future. This should be achieved within a reasonable timeframe, so that services can start to be built and benefits can be quickly shown. All the remaining strategic tasks should continue to be addressed in parallel with the ongoing tactical service implementations.

The prescription above will not cure all of your SOA ills but will introduce a dose of prevention for the future. Building services following a consistent set of standards, using a consistent set of tools, and deploying on a consistent platform from the very beginning will ensure the success of your whole SOA program, not just a few projects or services.

SOA and EDA - One and the Same

Event-Driven Architecture (EDA) has received a lot of coverage lately. Gartner’s analysts spent a lot of time last year talking about it and promoting their ideas on how it relates to the Enterprise Architecture. A lot of companies, lead primarily by the financial institutions, have taken the plunge into the EDA. Alerts offered by virtually every bank are a good example of this. However, I do not feel that the relationship between EDA and SOA has been fully explored yet.

EDA is based on the concepts of events, publishers, and subscribers. At the most basic level, the idea behind EDA is that publishers publish events and subscribers consume them. Of course, some logic and rules must be applied to properly route the events. Through this mechanism, systems become connected in a loosely coupled fashion. This makes the integrations a lot easier and eliminates the need for each publisher and subscriber to know the details of how to communicate with each other.

Sounds familiar? Absolutely! Change the terms events, publishers, and subscribers to services, providers, and consumers respectively and the paragraph above reads like an explanation of SOA. Why is this, you would ask? Because EDA is nothing more than an asynchronous version of SOA. The major difference between the two is that services are typically implemented as real time calls while events are published and consumed asynchronously. All the other concepts are virtually the same.

While the architectural approaches and design patterns for EDA are slightly different from SOA, the fundamental concepts are still the same. A central event handling infrastructure that knows how to receive, route, and transform the messages is required. It should be viewed as practically the same thing as the Enterprise Service Bus (ESB). In fact, generically, I would call it the Enterprise Eventing Bus (EEB). As events are published, they need to be translated into a common representation, so that a consistent set of rules and operations can be applied to them. A canonical model is the best solution to achieve this goal. Additionally, the same façade pattern should be used as described in the SOA Façade Pattern post to abstract the publishers from knowing and being tied directly to the Enterprise Canonical Model. Note that the logical EDA architecture presented below is very similar to the one introduced in the SOA Façade Pattern.

Events, just like services, should be registered and be discoverable via a central Registry mechanism. Subscribers looking to consume specific events should be able to discover them at design time and receive them from EEB at the run-time. Design of the events, their publication and consumption should also follow the same standards and patterns as service design. All of the related documentation should be stored in the same centralized repository. Governance mechanisms applied to events should also be very similar to those applied to services. EDA does not require establishing a completely new governance model but can effectively leverage existing SOA governance practices.

The bottom line is that EDA and SOA are virtually the same architectural approaches. If you think about EDA as an asynchronous version of SOA, everything else will fall in place.

SOA for the REST of Us

There has been a lot of discussions comparing REST with SOAP and presenting various points of view on how these technologies affect SOA. The opinions expressed are widely varied. Some believe that REST is the best thing since sliced bread because it reduces load on the network and eliminates the unnecessary complexity of the WS-* standards. Others argue that REST cannot yet be effectively used with SOA due to the protocol’s relative immaturity. More and more, however, we are hearing voices of moderation that consider both approaches complementary.

I believe there is place for both REST and SOAP in an SOA program. You cannot necessarily prohibit the use of one technology over the other with one notable exception – you cannot yet standardize solely on REST. SOAP should still be considered the preferred protocol with REST being utilized in very specific situations. There is a number of reasons why.

1. Enterprise applications require enterprise capabilities
   REST was designed for simple Web-based interactions, not for complex enterprise applications. A plethora of WS-* standards exist specifically to address the complexity of enterprise integration and interaction needs. Capabilities such as transactions, security, policy, guaranteed delivery, and many others are a must in any enterprise caliber system. REST does not yet support this level of standards and, most likely, never will due to its focus on simplicity and performance.


2. Security
   REST does not support any specific security standards. In fact, it relies on the infrastructure and middleware to secure end-to-end communications. If service consumers have more robust and complex security requirements than can be met with the underlying infrastructure alone, REST becomes insufficient.


3. Strong contracts
   Strong adherence to service contracts is the cornerstone of any SOA implementation. It ensures that services expose well-defined contracts and provide adequate information on how to access them. SOAP-based web services have built-in capability to validate their contracts. REST does not inherently support this type of verification. In fact, all the interactions between the consumers and RESTful services contain nothing more than a command and a list of parameters. It becomes the responsibility of the service to ensure the validity of the contract.

As a general rule, REST should be used when performance and simplicity are paramount, no special security requirements exist, and no complex interactions between the consumer and the service are necessary. The best uses for RESTful services should be considered simple extracts of small data sets, inquiries against open public data sources, calls to external vendors supporting the protocol, etc.

A number of SOA and middleware vendors are feverishly working on creating solutions that support both SOAP-based and RESTful services. Very soon, REST and SOAP will simply become some of the many communication protocol choices existing in the rich SOA toolbox. Most of the differences will be eliminated by a combination of server-side technologies and client-side frameworks. However, until this happens, strict standards should be established guiding the use and adoption of RESTful services.

SOA and OO

While SOA and OO are based on some very similar concepts, the approaches they utilize to achieve them are quite different. The biggest difference lies in the distributed nature of services. OO technology assumes local, in-memory calls while SOA assumes a distributed network of services. This fact drives a number of differences between the two technologies.

One of differences is the cost of traversing the distributed network boundaries. It is nontrivial in terms of complexity and performance. Service-oriented designs acknowledge these costs by putting a premium on boundary crossings. Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation. Compared to distributed objects, the service-oriented model views cross-service method invocation as a private implementation technique, not as a primitive construct — the fact that a given interaction may be implemented as a method call is a private implementation detail that is not visible outside the service boundary.

Another difference is autonomy of the code. Object-oriented programs tend to be deployed and act as a unit. They are not autonomous but rather embedded into a managing container. Service-oriented development departs from object-orientation by assuming that atomic deployment of an application is the exception, not the rule. Services are deployed, managed, and run as autonomous units. They are, in fact, the containers, in which OO code lives.

The final difference is the architecture. Since OO almost always deals with in-memory calls, it places no premium on the number of method invocations and the amount of data passed. In fact, the best OO interface contains many methods that are created for a single purpose. Under SOA, network overhead needs to be considered. This fact puts a premium on the amount of calls made from the client to the service. Thus, the number of service calls should be limited while the amount of data passed should be maximized.

Keep in mind that OO and SOA are not competing but rather complementary approaches. Think about service as an outer shell of an OO application that enables it to become sharable across the network.