The Architect Manifesto

IT architecture is a well defined profession. It was in existence ever since computers were first created in the 1940s and software needed to be developed for them. While it became more formalized only in the 1980s, the profession took huge steps forward in the last 30 years. Today, the world of IT architecture includes such disciplines as SOA, integration architecture, system architecture, data architecture, security architecture, etc. Yet, despite all of its different facets, IT architecture core practices, goals, and approaches remain the same across all of its disciplines.

There are several key principles that help us become successful as IT architects. No matter what problems we face, no matter what types of solution we need to apply, no matter what technologies we use, these general guidelines should always be followed. This forms the basis for the Architect Manifesto.

My research indicates that there have been multiple attempts at defining an Architect Manifesto. None of them seem to have succeeded or gained traction. Yet, I am certain that we, as architects, need the guiding light of a set of principles that will guide us in everything we do. We already unconsciously understand and follow them, but a formal definition will strengthen and inspire us. This is my attempt at verbalizing what we already know and apply in everyday work – the Architect Manifesto.

We, IT architects, guide the design and evolution of computer systems. We strive to achieve maximum value for our business partners through this work. We constantly discover better ways to architect systems. As a result, we have come to value: Simplicity over complexity Pragmatism over perfectionism Thinking out of the box over applying the same approaches to every problem Developing technology agnostic solutions over making technology driven decisions Delivering solutions to the business problems over concentrating purely on technology deliverables Looking at the big picture over getting too deep into details Long term, strategic thinking over short term, tactical thinking That is, while there is a place for the tactics on the right, we should strive towards approaches on the left.

I would appreciate feedback from all of the architects out there to make this manifesto more complete and relevant.

Architectural Thinking

I recently had to give an Architectural Thinking course. As I was putting the materials together, I wondered – what really makes a good architect? What is the difference? Is it personality, totality of experiences, different wiring, or natural inclination?

I always believed that while good architecture skills can be taught, it is the way we think that differentiates us and makes us good architects. Our natural ability to understand the problem and determine the right solution is the magic formula. This – and this alone – distinguishes good architects in our midst. You don't have to have "architect" in your job title or even be in IT to be a good architect. You can understand the technology in minute detail, know all the design patterns, possess tremendous depth of experience, have in-depth knowledge of various design techniques, etc. All this can be taught or acquired. At the end of the day, however, it is your approach to problem solving and the way you think that will make you stand out among your peer architects.

Why do I put such a heavy emphasis on the thinking style? To help you understand, we need to take a step back and define "architecture". According to Wikipedia, "the software architecture of a program or computing system is the structure or structures of the system, which comprise software components, the externally visible properties of those components, and the relationships between them." Many books, articles, and white papers provide a definition of architecture as well. However, in my opinion, these definitions are too complex and do not truly reflect the nature of architecture. My personal definition is very simple: architecture is a high level view of a technical solution to a business problem.

What does the definition of architecture have to do with our approach and thinking style? Everything! Taking all the specifics of how we actually deliver architectural solutions out of conversation, to do our job well we simply need to determine the right technical solution to the right business problem. We need to consider all the possibilities when addressing a business problem – what is there today, what is missing, what problem are we really trying to solve, is the current process the most efficient, what would deliver the most value, are we solving the right problem?.. We should be able to look at the problem from a very high level, abstract ourselves from the underlying technology or processes, and envision the most effective and efficient solution. As in the old adage, we should see the forest for the trees. Once the "ideal" solution is found, we can start concentrating on its details and understanding technology implications. Many times, we will discover that our vision cannot be readily implemented due to technology limitations, insufficient process maturity, and a host of other factors. Do not despair, however. Your vision becomes the solution goal state, and you would need to create a roadmap to get there.

Undoubtedly, some of you will disagree with the approach presented above. I can hear the arguments now – "You must consider the current situation to determine the right solution to the problem", "The project has a limited scope, and thinking broader is impractical", "The technology landscape should be considered upfront to design the most effective solution", etc. However, if we are to find the right technical solution, we must shed the old baggage. It is very hard to find new and innovative solutions if you cannot think outside the current box. To a large extent, the architectural thinking principles are grounded in the definition of what makes a good architect.

Architecture is an art, not a science. Therefore, a good architect is more of an artist – creative, imaginative, someone who can paint with a big brush. In my opinion, the characteristics and approaches listed below are the true differentiating factors among architects.

• Abstract thinking – this is the #1 quality of a good architect. You must be able to see the big picture and understand it abstractly, absent of many details that can cloud your judgment.


• Out-of-the-box thinking – many situations require us to be creative and innovative in our approaches. Good architects should be able to adapt to new situations easily and come up with the right solutions regardless of the situation.


• Clarity of vision – you, as an architect, should be able to clearly envision the solution and all of its implications including business process, technology, low level design, development, and potential phased delivery.


• Strength of convictions – architects should always try to do things right the first time, oppose inappropriate or wrong decisions, and stand up for what they believe are the right architectural solutions.


• Critical thinking – architects should always cast a critical eye towards their domain. You should challenge everything. Nothing should remain status quo or off limits. There are always opportunities for improvement. Don’t miss them because you feel comfortable with the current situation or are used to doing things a certain way.


• Problem solving skills – architects are problem solvers. Good architects strive to solve problems in the minimalist way, i.e. reaching the right solution in the most efficient manner. Even better architects ensure that they are solving the right business problem.


• Soft skills – this one is obvious. Good architects should have excellent soft skills to work well with the diverse audiences they are exposed to every day.

As in the everlasting nature vs. nurture debate, I believe good architects cannot be made – a large portion of what makes architects stand out is ingrained in how we think, act, and approach problems. To be truly effective, we should practice all the elements of architectural thinking and exhibit all the traits of a good architect.

Cloud Computing and the Reality

Cloud computing is all the hype nowadays. All you hear from vendors, analysts, and consulting companies is that cloud computing will solve all of your IT problems. Here are just a few promises associated with cloud computing:

• Eliminate your data center
• Solve all of your scalability and on-demand computing challenges
• Simplify infrastructure
• Reduce IT spend
• Make IT operations more efficient

Are they all true? It’s a possibility. To determine what cloud computing may mean to you, examine how it fits into your IT strategy and the way you deliver technology services to the business. Here are a few things to consider.

First of all, everyone needs to understand what cloud computing really is. According to Wikipedia, “Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet”. (See http://en.wikipedia.org/wiki/Cloud_computing.) Too many people, however, forget that it is only a style and begin to associate cloud computing with specific product offerings such as the Amazon Elastic Compute Cloud (Amazon EC2), Google Apps, Microsoft Azure, and others. Companies are not limited to just third party solutions. They can implement their own private clouds if they choose.

Secondly, you need to understand the vision behind cloud computing. The idea is simple – to seamlessly provide flexible, on-demand computing resources whenever necessary. This is not a revolutionary development. The Application Service Provider (ASP) model has been in existence for years. Infrastructure outsourcing practices have been utilized a long time before cloud computing became a term. So, what is all the hype then, you ask. They keys are the ubiquitous nature of the protocols used, increased reliability of the Internet, and the packaging of the offering as a generic service. Cloud computing, as a general approach, may support outsourcing of specific applications, generic computing resources or platforms, and software services. It may potentially lead to outsourcing of the whole data center.

Finally, all the pros and cons behind cloud computing need to be considered. Having someone take care of all your computing resources without investing into expensive data centers is an appealing concept but loss of control and unreliable SLAs may be a cause of concern for a number of businesses. Since the Internet is the primary communication mechanism for the public cloud, its reliability and performance need to be questioned whenever considering third party cloud offerings. Private clouds provide better control, reliability, and performance but what is the real difference between those and existing data centers? In my opinion, aside from following a different architectural model of allocating computing resources, nothing. On-demand computing is a great concept but making it work effectively is a tough task. Technologies exist today to dynamically divert unused resources to those applications that need them most. Grid computing, virtualization platforms, and others all provide these capabilities. However, there are limitations. Whenever maximum capacity is reached, hardware needs to be added. No software trick will work to cover this up. Therefore, efficient capacity and pipeline management need to exist to make cloud computing an effective and viable platform.

While there are some cloud computing zealots (http://www.infoworld.com/d/architecture/soa-realized-enterprise-computing-cloud-computing-146) and realists (http://www.gcn.com/Articles/2009/03/09/Guest-commentary-SOA-cloud.aspx), many are still cautious about this technology. And for a good reason. In my opinion, cloud computing has proven its worth in a number of situations but it is still not ready for the enterprise. Public clouds are too fickle for really demanding applications. Private clouds have not yet been effectively built. More importantly, however, lack of cloud computing standards and consensus among the key players will present challenges for anyone trying to enter this arena.

SOA Ecosystem

In the previous post, I’ve discussed the role of the ESB in the SOA ecosystem. However, I failed to adequately describe what an SOA ecosystem was and what were all the inter-relationships that must exist in order to make it truly effective.


If you refer to the diagram above, you will notice several major components that make up the SOA Ecosystem.

1. ESB
2. Registry/Repository (RegRep)
3. Security
4. Service Management
5. Shared Service Environments
6. Service Consumers
   

The SOA Ecosystem also encompasses all the related elements such as the application and service developers and testers as well as all the tools being used to accomplish these activities.

To truly comprehend how the SOA ecosystem operates, a clear understanding must be developed of what each component does and what its role is. Let’s start from the service consumer side.

• Service Consumers
  
  • Application Developers build applications that consume services. They use IDEs and other development tools to construct service requests and parse responses. Developers interact with the Registry/Repository to find the right services, obtain service metadata, and understand usage patterns.
  • Application Testers perform quality assurance tasks on the final product.
  • Application Servers that execute the application code interact directly with the SOA platform hosting the services.
• SOA Infrastructure
• Service Management Platform acts as an entry point into the SOA infrastructure. It retrieves policy information about the service being executed and applies it appropriately to the request. The policy is used to understand service security and authority, associated SLAs, constraints, contracts, etc. The Service Management Platform is often utilized to keep track of the service consumption and run-time metrics, which are then fed into the Registry/Repository.
• The role of the Enterprise Service Bus has already been discussed.
• Registry/Repository acts as a central repository for services and their metadata. Its uses and integrations are discussed at each related point.
• Security / Authentication Platform is a part of the larger IT infrastructure and is typically represented by either LDAP or Active Directory technology.
• Shared Service Environments are used to host reusable services. While different organizations choose to approach service hosting differently, if a common service hosting platform can be established, many issues related to service scalability, performance, reuse, security, implementation, standardization, etc. can be easily resolved. A centrally managed platform can be easily upgraded to accommodate additional – foreseen or unforeseen – volume. Standard capabilities can be provided to perform security, authentication, logging, monitoring, instrumentation, deployment, and many other tasks.
• Service Creation
• Service Architects and Developers create reusable services using the appropriate design and development tools. They also interact with the Registry/Repository to discover existing services and register new services and related metadata. The created services should ideally be deployed into a Shared Service Environment.
• Service Testers perform quality assurance tasks on the new or modified services. They use special SOA testing tools to create test cases and automate their execution. These tools interface with the Registry/Repository to retrieve metadata about the services and update related information once testing is complete.
  

Many vendors have slightly different view of how their products integrate and interact with the SOA ecosystem, what components exist, and where they are located. One can also include other elements into the SOA ecosystem such as the EA Repository, CMDB, IT Governance Tools, Monitoring Tools, etc. However, the general dynamics remain the same. The key point is that the SOA infrastructure interacts with the rest of the IT platforms and people in a certain way. In order for SOA to be truly successful, these interactions must become natural, automated, and symbiotic.

To ESB or Not to ESB?

That is the question. Many SOA thought leaders have addressed this topic. Most recently, David Linthicum wondered if ESBs were evil. He also talked about ESBs hurting SOA in his blog. Eric Roch has chimed in on the debate by providing some general guidelines for how to use the ESBs. Joe McKendrick has summarized the recent debate in his blog.

There seems to be a lot of pent up emotions in the industry when it comes to the ESBs. A lot of people tend to view ESBs as over-engineered, complicated, and unnecessary. Maybe, it is a backlash from the vendor hype or consistent experiences with a failed ESB implementation. Maybe, it is a reaction to the industry’s push towards choosing the tools first and fitting the solution into them later rather than vice versa. Maybe, it is a response to the architects calling the ESB implementation Enterprise SOA. I don’t know. What I do know is that ESBs have its place and when properly used are very useful.

SOA in not just about exposing services via a ubiquitous protocol and letting people use them. A successful SOA must have the following elements in place:

• Governance and Processes
• SOA Governance
• SOA Methodology
• SOA Reference Architecture (and possible Reference Implementations)
• SOA Maturity Model
• Service testing and versioning approaches
• SOA design patterns
• Technology
  
  • ESB
  • Service Management platform
  • SOA Governance platform
  • Registry / Repository (often is part of the SOA Governance platform)
  • SOA testing tools

The diagram below depicts the preferred SOA ecosystem and relationships between all of its different components and actors.



Note that the ESB plays a central role in the SOA ecosystem. It needs to be tightly integrated with the Registry/Repository tool that will store policy information and service metadata, service management platform that will ensure compliance to the predefined policies, and platforms exposing the physical service endpoints. ESBs are very useful when utilized to perform the following tasks:

• SLA and policy management
• Security reconciliation
• Protocol reconciliation
• Message transformation
• Orchestration (possibly, in conjunction with a BPM tool)
• Integration
• Logging and instrumentation
• Metrics collection

ESBs can provide all these capabilities in a central location and in a consistent fashion, so that every service does not have to implement them individually. Every service has to perform each of these tasks in some way, shape, or form. Without a central tool, implementations, tools, and approaches will vary. At the end of the day, you will end up with a hodge-podge of different things, which will be hard and costly to maintain.

When services are created, it is impossible to know who and how will consume them. In fact, it should be irrelevant. Services should not worry about all of the potential consumers, protocols, and contracts. It is the job of the ESB to reconcile all of them. Services should not have to include all of this complexity in their designs and implementations. They should only make sure that the business logic is properly implemented and a standard interface is provided. The ESB will take care of the rest.

Obviously, without proper planning and architectural oversight, ESBs can fail. Using an ESB to support only a handful of services is an overkill. Blindly choosing a product without performing adequate analysis always leads to problems. However, putting ESBs in the right place in the SOA ecosystem and utilizing them for the right purposes will only simplify the development, increase efficiency, clearly distribute the responsibilities between architectural components, and improve standardization. ESBs are not evil when used correctly.

Context Delivery Architecture

At the last Gartner Application Architecture, Development, and Integration (AADI) Summit held in Orlando, FL at the beginning of June, William Clark introduced the concept of Context Delivery Architecture (CoDA). It is an architecture that is aware on the end user’s context such as location, preferences, identity, etc. and delivers the information that is most suited for it. Another way to describe it is WYNIWYG (what you need is what you get) services. The basic idea is that users’ specific context will drive what information they receive, how the applications interact with them, and where the processes take them.

Despite Gartner labeling CoDA as an emerging trend, I don’t believe it is a new concept. Context and location aware applications have been in existence for a while. Think back to the Internet bubble days when all kinds of schemes were designed to deliver coupons, advertisements, and other “useful” information to your mobile devices when you got close to certain location. RFID and its applications became the staple of context aware applications. Even Gartner based its research on these trends. It is true, however, that CoDA has not yet become mainstream and is moving up the Gartner hype cycle curve.

CoDA is still very immature. The vision is that CoDA applications will ubiquitously run on a variety of devices, technologies, and platforms. For this to become a reality, technology needs to be created that would allow the same services to be delivered to a variety of platforms that possess the same context aware capabilities. Users should benefit from being mobile, not be hampered by it. For example, salespeople that left the office for the client visit should be able to obtain specific customer information, find out sales status, and view the whole relationship picture immediately on their preferred device. The same capabilities should exist on all mobile platforms, which will truly make context aware applications possible. At the same time, mobile devices should evolve to ubiquitously interact with the network. Whether a WiFi, cellular, or any other kind of network is available should not prevent the application and the device from performing their functions.

Even though CoDA was billed by Gartner as the next step in the evolution of SOA, I don’t think it fits into the same paradigm. SOA’s primary goal is to create composite applications through the leverage of existing services. EDA, or as Gartner likes to call it, Advanced SOA, pursues the same objective, except that instead of services, the same events are sought to be consumed. By contrast, CoDA aims to enhance user’s experience through the knowledge of his/her context and tailor the application behavior to it. While it builds on the concept of reusable services that would deliver the right information at the right time, the whole concept has nothing else in common with SOA. In my opinion, CoDA is a move towards more intelligent applications but it is definitely not the next evolution of SOA.

CoDA still has a long way to go. It is an exciting concept that has science fiction written all over it. However, the technology, devices, networks, and people are nearing the point when context aware applications will become commonplace. The exciting thing is that I don’t think we have much longer to wait.

Starting on Your SOA Journey

Many industry leaders believe that SOA should be started small and evolve over time. The argument is that this approach gives companies an opportunity to implement SOA practices on a small scale, test them out in a controlled environment, and understand how everything would work within the organizational boundaries. This is not a bad idea. However, I believe that the most effective way to introduce SOA is to build out the whole infrastructure, introduce the necessary technology, and establish all the patterns, best practices, reference architectures, and governance mechanisms before creating a single service.

The reasoning for this is simple. SOA on a small scale is not SOA. It is just a bunch of services. SOA’s goal is to create and leverage services across the organization. A single project or a couple of services cannot achieve this. Furthermore, effective governance, best practices, and lifecycle processes cannot be established on a small scale. They need to be designed and implemented with the large scale in mind. Testing them on a single project is not only impractical – it doesn’t provide any knowledge of how SOA will truly work within the organization.

Any successful SOA implementation will eventually have all of its elements in place – infrastructure, technology, governance, practices, processes, and people. Consider the impact of growing all this organically. You will end up with a hodge-podge of services implemented on different platforms using different approaches and design patterns. The technology set will be inconsistent. Governance mechanisms that typically tend to be established late in the game will most likely allow inadequately designed and implemented services to go into production. All this would have to be remediated at some point of time. Imagine the effort required to clean up years of organic growth! Most companies simply move on and leave the mess behind.

Now imagine what will happen if all of the SOA elements are in place from the very beginning. No rework, re-platforming, or cleanup will be required. All of the services will reside on the right platform that can be scaled for future demands, all of the best practices will be followed, and the governance mechanisms will be able to catch most, if not all the subpar services. The company will be able to reap SOA benefits right away without having to do the costly cleanup or conversion.

Of course, waiting to complete all the preliminary work can take years. No company, regardless of how strong its commitment to SOA is, can wait that long to start seeing the benefits of something that will require a lot of upfront investment. Thus, the most pragmatic approach is to introduce as many SOA elements as possible that will provide the most complete and consistent SOA foundation for the future. This should be achieved within a reasonable timeframe, so that services can start to be built and benefits can be quickly shown. All the remaining strategic tasks should continue to be addressed in parallel with the ongoing tactical service implementations.

The prescription above will not cure all of your SOA ills but will introduce a dose of prevention for the future. Building services following a consistent set of standards, using a consistent set of tools, and deploying on a consistent platform from the very beginning will ensure the success of your whole SOA program, not just a few projects or services.

SOA and OO

While SOA and OO are based on some very similar concepts, the approaches they utilize to achieve them are quite different. The biggest difference lies in the distributed nature of services. OO technology assumes local, in-memory calls while SOA assumes a distributed network of services. This fact drives a number of differences between the two technologies.

One of differences is the cost of traversing the distributed network boundaries. It is nontrivial in terms of complexity and performance. Service-oriented designs acknowledge these costs by putting a premium on boundary crossings. Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation. Compared to distributed objects, the service-oriented model views cross-service method invocation as a private implementation technique, not as a primitive construct — the fact that a given interaction may be implemented as a method call is a private implementation detail that is not visible outside the service boundary.

Another difference is autonomy of the code. Object-oriented programs tend to be deployed and act as a unit. They are not autonomous but rather embedded into a managing container. Service-oriented development departs from object-orientation by assuming that atomic deployment of an application is the exception, not the rule. Services are deployed, managed, and run as autonomous units. They are, in fact, the containers, in which OO code lives.

The final difference is the architecture. Since OO almost always deals with in-memory calls, it places no premium on the number of method invocations and the amount of data passed. In fact, the best OO interface contains many methods that are created for a single purpose. Under SOA, network overhead needs to be considered. This fact puts a premium on the amount of calls made from the client to the service. Thus, the number of service calls should be limited while the amount of data passed should be maximized.

Keep in mind that OO and SOA are not competing but rather complementary approaches. Think about service as an outer shell of an OO application that enables it to become sharable across the network.

Creating and Managing a Canonical Model

As I discussed in my previous post (http://leoshuster.blogspot.com/2008/05/soa-canonical-modeling.html), a canonical model is a critical element in any successful SOA program. However, given the fact that it should describe the business completely and in a standard way, the resulting model will most likely be very large and complex. Thus, a natural question begs to be asked: how do you create and maintain a canonical model?

There are basically three ways to create a canonical model.

1. Buy it or adopt an existing industry standard model
   There is a number of organizations that either developed a set of standard models targeting a specific vertical (e.g. IBM - http://www-03.ibm.com/industries/financialservices/doc/content/bin/fss_ifw_gim_2006.pdf) or maintain industry standard definitions (like MISMO for Financial Services or ACORD for insurance). You can adopt one of these models to serve as the canonical representation of all the business entities.




Pros:



• Most of the work is already done
  
• Another organization maintains the model for you and introduces changes as necessary
  
• The model is standard and should help with external partner integration
  


Cons:



• The specifics of your organization may not be complete captured, which require custom additions to be made
  
• Some changes or customizations may be needed that would make it harder to upgrade in the future
  
• The elements are too generic or unnecessarily complex
  
• High learning curve for canonical model consumers
  






2. Create it from scratch
   A canonical model is created from scratch and built out completely before any work utilizing it can begin. This would require at least 3-6 months of effort meeting with various groups across the organization, collecting and sorting the information, and validating the result with the potential users.




Pros:




• Would provide the most complete and targeted model
  
• Users will have innate knowledge of the model since they helped build it
  


Cons:




• Requires all projects that need to use the canonical model to stop until it is completed
  
• Not highly realistic or pragmatic approach
  
• Requires modifications to be made and managed internally
  






3. Build it incrementally
   A canonical model is built incrementally over the span of multiple projects. Only those elements that are required by the project are added or modified.




Pros:




• Does not require a lot of upfront effort to get started
  
• Efficient and demand-driven – model only what is needed
  
• Low learning curve – users have more opportunity to learn it as the model evolves
  


Cons:




• High propensity for change – the model is frequently refactored as new projects leverage it
  
• Requires a centralized team to own or govern it
  
• Frequent changes would require a large amount of testing and updates to the existing consumers
  

The best way to manage the changes to the canonical model is to establish a centralized team to own the whole thing or to provide governance over it. It would be responsible for making / tracking the changes, notifying consumers, performing compatibility testing, versioning, training, and communications. A comprehensive list of all the canonical model consumers needs to be maintained in order to notify them of all the relevant changes and understand the overall impact of modifications. Without a centralized team, there can never be a canonical model because there will be no one to synchronize or drive all the disparate efforts towards a single goal.

Regardless of the chosen approach, changes to the canonical model are inevitable. Therefore, the façade pattern described in one of my earlier posts must be utilized when using canonical models in the SOA context.

SOA Façade Pattern

In the last post, I discussed the value of the canonical modeling and described how to minimize the impact of canonical model changes on the service consumers. The solution was to use the façade pattern. I would like to elaborate on this topic since a more in-depth discussion is needed to define the pattern and understand its uses.

A good definition of the façade pattern can be found on Wikipedia: http://en.wikipedia.org/wiki/Facade_pattern. In general terms, it is described as “a simplified interface to a larger body of code”. This is exactly how it should be applied to SOA. A façade should be built in front of any service whose interface is based on the canonical model. Consumers would not access the service directly but rather through its exposed façade interface. In fact, the canonical interface should only be exposed for internal consumption. Each façade should be designed to be specific for each consumer or a group of consumers and not directly tied to the canonical model. The diagram below depicts the pattern details and its usage.

There are several distinct benefits of using the façade pattern.

1. Façade shields service consumers from the changes in the canonical model.
   If every consumer was dependent on the canonical model, even the smallest change could have disastrous effects. All of the services as well as potentially all of the service consumers would need to be re-tested. Lacking automated regression and functional tests already developed, this would be a major undertaking. Using the façade pattern would minimize the impacts of any canonical model changes. Since the facades are specific to each consumer and are not directly tied to the canonical model, the only thing that would need to change is internal mapping between the façade interface and the canonical model.
   
2. Façade hides the complexity of the canonical model.
   Modeling the whole business domain is not a simple task. Therefore, canonical models are usually large and complex. Service consumers do not typically want to know the entire canonical model and understand all of its intricacies. They want to get the data they need and continue performing their business functions. Exposing a consumer-specific interface via the façade prevents service consumers from having to know any canonical model details. Additionally, since canonical models are fairly generic, most of the data elements in the returned entity may not be relevant to the consumer. A façade simplifies the request and response data structures and ensures that only relevant information is returned.
   
3. Façade returns data representation understood by the consumer.
   A canonical model is generic. It is designed to describe the whole organization. However, service consumers typically operate in their own specific domains. Service façade that is designed to return data in a format that consumers understand simplifies the overall consumption experience and reduces the overall efforts. The consumer does not need to perform any translations and can start working with the data right away. Additionally, a façade can help representing the same entity differently for different consumers if so required. There may be instances, for example, when one Line of Business (LoB) thinks of a customer one way while another LoB views a customer completely differently. These views may even be largely incompatible but as long as they are represented in the canonical model, a façade can be created to address specific LoB needs.

The façade pattern introduces a small translation layer between the service consumer and the canonical service interface. It should not contain any generic business logic but can perform some consumer-specific operations. While it may cause reduction in performance and increased development costs, the negative impacts should be minimal and will be offset by the benefits described above.

SOA & Canonical Modeling

There has been a number of discussions in various articles and blogs on the topic of application and value of canonical modeling for SOA. Majority of industry experts support the use of canonical models as one of the key SOA patterns. However, there are some that consider canonical modeling as detrimental to SOA (http://service-architecture.blogspot.com/2006/08/single-canonical-form-not-for-soa.html). I would like to add my voice to those that consider canonical modeling a critical aspect in the success of the SOA program.

Primary goal of any SOA program is to introduce a variety of reusable services. Reusability typically implies that a service has a number of different consumers and data providers. The first and most obvious value of a canonical model is that it acts as an abstraction layer between all those consumers and providers. It is an old and well-known design pattern – when you have a number of data sources and their consumers that need to be integrated together, you introduce an abstraction layer, so that neither is aware of the internal details of the other. This way, any changes made to a consumer or a provider will have minimal impact on all of its integration points. This is the second benefit of using a canonical model. It minimizes the impact of internal service changes, modifications of data sources, or switch to a new backend data source on the service consumers. The canonical model should remain unchanged regardless of what happens inside the service, which, in turn, ensures that the contract between the service consumer and the service itself remains unaffected. The maximum possible impact on the service may be the need to change the mappings between internal service data structures and the canonical model.

Since the canonical model minimizes the impact of internal service changes on its consumers, it also reduces the need for regression testing. (This is the third benefit of canonical modeling.) If services did not provide a layer of abstraction between its internal implementation, backend data sources, and its consumers, any change inside the service or data provider would be reflected in its interface and thus would require a full regression test. Usage of a canonical model eliminates the need to perform rigorous regression testing since, as we discussed above, any such changes would not impact the service consumer. The only thing that would need to be done is perform a test validating that the service contract did not change.

Representing standard structures in a canonical model maximizes service reuse (fourth benefit). Consider all the entities that your business deals with every day. It could be customer, account, price, payment, etc. However, different parts of the organization may view these entities differently. One division, for example, may care about the customer household information while other about his/her geo-positioning. Without a canonical model, you would end up with multiple slightly different representations of the same entity. This results in services being built based on disparate models targeted for only specific audiences. Other groups trying to reuse these services would require changes to address their needs or would simply not be able to consume them. Representing all entities in a standard way eliminates this incongruence and allows different parts of the organization to speak the same language. This, in turn, maximizes the potential and real reuse of services built across the company.

Of course, some would argue that any changes to the canonical model would impact all of the service consumers and they would be right. In order to minimize this risk, a façade pattern needs to be used. Rather than exposing the canonical interface directly to the consumers, a façade would need to be build based on each consumer’s needs. It would expose data contracts specific to each consumer or a group of consumers. The service would never be called directly but only through one of its façade interfaces. This way, any changes made to the canonical model would not impact the service consumers directly. The façade would remain intact. The only change that would potentially need to be made is modification of mappings between the façade and canonical data structures.

Used together, the canonical modeling technique and the façade pattern will maximize the service reuse and minimize the impact of internal changes on service consumers. The approach will save costs and time on regression testing efforts. Regardless of what the opponents say, use of these techniques is critical to the overall SOA program success.

Links to some good SOA & canonical modeling articles:
http://www.it-eye.nl/weblog/2007/06/13/soa-best-practice-9-use-a-canonical-data-model/
http://www.ibm.com/developerworks/db2/library/techarticle/dm-0803sauter/
http://www.soapatterns.org/canonical_interface_expression.asp
http://reallifeserviceorientedarchitecture.blogspot.com/2007/11/how-to-create-canonical-form.html

SOA & Agile Software Development

Many industry articles and experts would like you to believe that SOA and Agile software development methodologies (let’s call them “ASDM” for simplicity sake) are made for each other. People like Gregor Hohpe of Google or Brenda Michelson of ebizQ would like you to believe that Agile and SOA are a powerful combination. I would like to set the record straight once and for all. ASDM, regardless of the flavor or the sort, is the worst thing you can apply to your SOA program.

Agile is created to build software quickly while requirements continue to shift thus allowing the end users to see the results in days rather than months. Refactoring (read “change”) is assumed as normal and is, in fact, welcomed. Software design is often organic and evolves with each story card or iteration.

SOA is an architectural style that requires rigorous planning, forethought, and discipline. Most experts will tell you that A (“Architecture”) is the important element in SOA. Services must be designed with an eye towards the future reuse, not immediate requirements. Contract-first development, which is inherent in any SOA approach, is largely alien to an ASDM. Same is true for comprehensive design cycles that focus on designing reusable, flexible, and architecturally sound services. Agile simply has no room for design. Code is self documenting, Agilists will tell you. Some Agile flavors that account for some design time still focus on a very narrow set of requirements and never take the big picture into account.

The most telling example of incompatibility between SOA and Agile is a project that needs to build a large number of shared services for a large number of consumers. Under an ASDM, each service would be built incrementally, over time as the story cards with new requirements come up. As new consumer’s requirements are satisfied, the service must change, which most likely triggers the need to test the impact on the existing consumers. The more consumers the service has, the more testing must be done. Using a model-driven design and contract-first development would undoubtedly solve this problem. Service interface would be modeled in its entirety and a complete contract presented to each consumer. This would eliminate the need to retest each subsequent consumer integration.

To take this example even further, imagine that this needs to be done for several, not just one project! And timelines are not evenly in synch. Amount of refactoring and testing would become enormous. Throw in the typical governance processes each organization has in place, a registry that keeps track of each service’s lifecycle and policies, service management platform that must be integrated with it, an ESB through which services must be exposed, etc. and you will get an even better picture. Without proper planning, design, and architecture, your SOA program will not succeed. No Agile methodology applies the level of rigor and depth required to build truly reusable services. Stop using Agile before it’s too late for your SOA program!