Service Orchestration Guidelines

Many SOA articles, white papers, and vendor documents talk about “service orchestration”. But understanding of the underlying concepts and orchestration best practices remain elusive.

A quick Google search will produce a number of articles and links that discuss service orchestration and related topics. Most of them will talk about BPM engines, ESBs, and BPEL. This, unfortunately, pollutes the true definition of service orchestration and gives it a much more technology centric view.

In my opinion, Service Orchestration is an automated way to combine several services together to achieve new functionality. The end result is a new composite service that provides a distinct business capability and can be invoked independently. It must have all the appropriate attributes as discussed in my previous article.

Orchestration is a technology independent concept. It can be achieved via a descriptive language such as BPEL, built-in tools within a specific platform (ESBs typically provide their own orchestration mechanisms), or programmatically. Depending on your needs, situation, or technology available, the best way to perform service orchestration may be different. Here are a few guidelines to help you create service orchestrations faster and make them more flexible, maintainable, and scalable.

• Use the platform with built-in orchestration capabilities as your first choice
• Avoid implementing service orchestrations programmatically whenever possible
• Choose a platform or mechanism that can easily perform flow logic, result aggregation, message transformation, and business rule application
• Ensure the composite service fits the definition of a service, i.e. has all the attributes of a service

The rationale behind the above guidelines is very simple. You want to choose a platform that already provides most of the capabilities you will need when creating new service orchestrations. You will typically need to call several services, aggregate their results in some way or chain the calls together through some kind of logic, transform the end result to match the exposed contract(s), and return it. The less work you have to do and the more you can rely on the platform’s capabilities, the more efficient your orchestration will be. If you can complete your orchestration work through a visual interface and never see the code, you are on the right path. This way, you will spend less time maintaining the orchestration, it will be easier to make changes, and you don’t have to build all the necessary mechanisms from scratch.

Many would argue that a programming language will give you the most flexibility when implementing an orchestration. While this is true, the overhead is pretty large and efficiency is low. First of all, no programming language seamlessly integrates all the mechanism you need to create an orchestration, especially in a visual way. Secondly, every time an orchestration needs to change in some way, no matter how small, new code needs to be written, deployed, and tested. While the same steps need to be performed on any orchestration platform, the level of effort will be a lot smaller on full featured orchestration platforms.

When creating service orchestrations, it is important to maintain proper relationships between composite and atomic services. The diagram below shows which services should be allowed to interact with each other.

The following list details the rules and guidelines for establishing relationships between composite and atomic services.

1. Atomic business services should not call each other. Use orchestration to combine several business services together.
   The goal of service orchestration is to combine several services together through a series of logical steps and expose new capability to the consumers. Orchestration platforms, as discussed above, provide a lot of functionality to make this work easy and efficient. If individual services are allowed to call each other, they would not be taking advantage of the orchestration platform’s capabilities. Furthermore, when business services call each other, it establishes a tight coupling between them, which makes them less reusable and harder to maintain. Atomic business services should provide specific, well defined business capabilities and be reusable in their own right. Reliance on other services to complete work indicates plurality of purpose and lack of specificity.
   
2. Business services can call Utility services.
   While coupling services together should be avoided as much as possible, sometimes generic, low level functionality that needs to be invoked from a business service is exposed via utility services. It would be an overkill and sometimes even impossible to use an orchestration platform in order to allow business services to take advantage of such functionality as logging, retrieving or storing configuration information, and authorization.
   
3. Utility services cannot call Business services.
   Utility services should not be tied to any business processes or capabilities. Thus, a utility service calling a business service would violate this rule.
   
4. Business services cannot call Composite services.
   The logic behind this guideline is the same as in disallowing business services call each other. A composite service is also a business service. Thus, a business service calling a composite service should not be allowed.
   
5. Composite services can call other Composite services.
   Other composite services are allowed to participate in orchestrations. They should be treated as regular atomic services in this case.

Note that, even though atomic business services and composite services are, in essence, business services, they are different and guidelines provided above are not contradictory in their treatment. There are two levels at which they should be compared -- logical and physical. From a logical perspective, atomic business services and composite services are the same. They expose some kind of unique business capability and adhere to the service definition guidelines. From a physical perspective, however, they are different. Atomic business services, as opposed to composite services, rely solely on internal business logic and direct interaction with backend data sources to perform their work. Thus, by definition, atomic business services should not call other business services as part of their implementation. By the same token, since composite services already rely on other business services to complete their work, they should not differentiate between calling atomic business services or other composite services.

Service orchestration is a complex topic and might take a series of articles to discuss completely. However, the rules outlined above should establish a good foundation for creating and managing composite services.

Services Explained

Since the SOA term was coined, many discussions raged on what a service was from a business and technical perspectives. In this article, I offer my views on the topic.

There are several categories of services. Many leading SOA vendors and thinkers typically break them down into Business and Utility types. A Business service represents a business capability that is needed to complete a step within a larger business process. Examples may include retrieving customer information, making payments, or checking order status. Utility services represent a technical capability that is business process agnostic. Examples are e-mail, logging, or authentication services.

Services can be combined together to create composite services. This is called orchestration. An example of this can be a Money Transfer service that needs to debit one account and deposit money into another one. Composite services can also be categorized as Business and Utility. Best practices and general orchestration guidelines as related to orchestrations, atomic services, and their relationships will be discussed separately.

Regardless of the type, a service is comprised of three components.

• Interface
• This defines how services are exposed and can be accessed by its consumers.
• Interfaces are not limited to Web Services and can be represented by any remote messaging protocol.
• Contract
• This defines what services expect during the interaction with the consumer. Message structures, relevant policies, and related security mechanisms are all part of a contract.
• Contract defines a “legal” agreement on how the service and its consumers should interact.
• Implementation
  
  • This is the actual service code.

A service may have multiple interfaces. Different consumers may have the need to access the service via different protocols. For example, a Java consumer would like to access a service via a Web Service protocol while a Mainframe application can only use MQ Series. Even though a service may itself expose multiple interfaces, it is more effective to let the ESB platform handle this. For more details about the rationale behind this recommendation, see “To ESB or Not to ESB?" post.

A service may also have multiple contracts. I have recommended in the past that for a service to be maximally reusable, it needs to implement the Service Façade pattern (see “SOA Façade Pattern” post and “Service Façade” pattern). This pattern recommends that multiple different interfaces and contracts for the same service be created. The Concurrent Contracts pattern also addresses this issue.

It is important to understand that while the interface, contract, and implementation describe a service as a whole, they are not closely tied together. An interface is not dependent on the contract details and should not be tied to the specific messaging structure. The opposite is also true – the contract should not be tied to any specific communication protocols. Additionally, the implementation should be flexible enough to accommodate the potential for multiple interfaces and contracts. Ideally, however, I would recommend that a service expose only a single contract and interface and the ESB would take care of exposing additional endpoints and facades as necessary.