To ESB or Not to ESB?

That is the question. Many SOA thought leaders have addressed this topic. Most recently, David Linthicum wondered if ESBs were evil. He also talked about ESBs hurting SOA in his blog. Eric Roch has chimed in on the debate by providing some general guidelines for how to use the ESBs. Joe McKendrick has summarized the recent debate in his blog.

There seems to be a lot of pent up emotions in the industry when it comes to the ESBs. A lot of people tend to view ESBs as over-engineered, complicated, and unnecessary. Maybe, it is a backlash from the vendor hype or consistent experiences with a failed ESB implementation. Maybe, it is a reaction to the industry’s push towards choosing the tools first and fitting the solution into them later rather than vice versa. Maybe, it is a response to the architects calling the ESB implementation Enterprise SOA. I don’t know. What I do know is that ESBs have its place and when properly used are very useful.

SOA in not just about exposing services via a ubiquitous protocol and letting people use them. A successful SOA must have the following elements in place:

• Governance and Processes
• SOA Governance
• SOA Methodology
• SOA Reference Architecture (and possible Reference Implementations)
• SOA Maturity Model
• Service testing and versioning approaches
• SOA design patterns
• Technology
  
  • ESB
  • Service Management platform
  • SOA Governance platform
  • Registry / Repository (often is part of the SOA Governance platform)
  • SOA testing tools

The diagram below depicts the preferred SOA ecosystem and relationships between all of its different components and actors.



Note that the ESB plays a central role in the SOA ecosystem. It needs to be tightly integrated with the Registry/Repository tool that will store policy information and service metadata, service management platform that will ensure compliance to the predefined policies, and platforms exposing the physical service endpoints. ESBs are very useful when utilized to perform the following tasks:

• SLA and policy management
• Security reconciliation
• Protocol reconciliation
• Message transformation
• Orchestration (possibly, in conjunction with a BPM tool)
• Integration
• Logging and instrumentation
• Metrics collection

ESBs can provide all these capabilities in a central location and in a consistent fashion, so that every service does not have to implement them individually. Every service has to perform each of these tasks in some way, shape, or form. Without a central tool, implementations, tools, and approaches will vary. At the end of the day, you will end up with a hodge-podge of different things, which will be hard and costly to maintain.

When services are created, it is impossible to know who and how will consume them. In fact, it should be irrelevant. Services should not worry about all of the potential consumers, protocols, and contracts. It is the job of the ESB to reconcile all of them. Services should not have to include all of this complexity in their designs and implementations. They should only make sure that the business logic is properly implemented and a standard interface is provided. The ESB will take care of the rest.

Obviously, without proper planning and architectural oversight, ESBs can fail. Using an ESB to support only a handful of services is an overkill. Blindly choosing a product without performing adequate analysis always leads to problems. However, putting ESBs in the right place in the SOA ecosystem and utilizing them for the right purposes will only simplify the development, increase efficiency, clearly distribute the responsibilities between architectural components, and improve standardization. ESBs are not evil when used correctly.