Thursday, May 1, 2008

SOA & Agile Software Development

Many industry articles and experts would like you to believe that SOA and Agile software development methodologies (let’s call them “ASDM” for simplicity sake) are made for each other. People like Gregor Hohpe of Google or Brenda Michelson of ebizQ would like you to believe that Agile and SOA are a powerful combination. I would like to set the record straight once and for all. ASDM, regardless of the flavor or the sort, is the worst thing you can apply to your SOA program.

Agile is created to build software quickly while requirements continue to shift thus allowing the end users to see the results in days rather than months. Refactoring (read “change”) is assumed as normal and is, in fact, welcomed. Software design is often organic and evolves with each story card or iteration.

SOA is an architectural style that requires rigorous planning, forethought, and discipline. Most experts will tell you that A (“Architecture”) is the important element in SOA. Services must be designed with an eye towards the future reuse, not immediate requirements. Contract-first development, which is inherent in any SOA approach, is largely alien to an ASDM. Same is true for comprehensive design cycles that focus on designing reusable, flexible, and architecturally sound services. Agile simply has no room for design. Code is self documenting, Agilists will tell you. Some Agile flavors that account for some design time still focus on a very narrow set of requirements and never take the big picture into account.

The most telling example of incompatibility between SOA and Agile is a project that needs to build a large number of shared services for a large number of consumers. Under an ASDM, each service would be built incrementally, over time as the story cards with new requirements come up. As new consumer’s requirements are satisfied, the service must change, which most likely triggers the need to test the impact on the existing consumers. The more consumers the service has, the more testing must be done. Using a model-driven design and contract-first development would undoubtedly solve this problem. Service interface would be modeled in its entirety and a complete contract presented to each consumer. This would eliminate the need to retest each subsequent consumer integration.

To take this example even further, imagine that this needs to be done for several, not just one project! And timelines are not evenly in synch. Amount of refactoring and testing would become enormous. Throw in the typical governance processes each organization has in place, a registry that keeps track of each service’s lifecycle and policies, service management platform that must be integrated with it, an ESB through which services must be exposed, etc. and you will get an even better picture. Without proper planning, design, and architecture, your SOA program will not succeed. No Agile methodology applies the level of rigor and depth required to build truly reusable services. Stop using Agile before it’s too late for your SOA program!

12 comments:

dix44040 said...

Very well said. To add to that, I think the scope and context of SOA projects is quite different from typical functional application projects. With typical functional application projects the business analysis can be based on a group of smaller users/experts that are well versed (and many times have hands on expertise) with the functional domain. With SOA projects, the user base that drives the requirements could be quite large (and with varied backgrounds) and thus - it could be more involved and necessary to firm up the requirements/contracts early on needing better planning. I think the definition of contract also differs significantly - SOA contracts could be more at a higher level whereas in functional applications these could be more granular and at multiple tiers - business service, portal, integration, etc. To take it further, In general, I think blindly applying one methodology across all types of projects is a big risk - the right methodology and governance should be applied based on organization, combination of in-house/external resources/expertise, and strategic objectives.

Leo Shuster said...

Very good points about the scope of typical SOA and Agile projects and the definition of contracts. I can't agree more with this assessment. To build truly reusable services, you need to consider all potential service consumers, which most often exist outside of a single application. Additionally, as I mentioned in the original post, the contract for a shared service must be well defined, which is a discipline Agile projects usually lack.

Very good point about a blind application of methodologies. I agree it is a big risk. Different projects can be good candidates for different methodologies. However, they will succeed only if the goals of these projects are aligned with the proper delivery mechanisms.

Daeron said...

Great discussion. I personally have had success with Agile and SOA and I combined them for fantastic and evolutionary deliveries. This is not to say that this is appropriate for journeyman level or below. Goal of SOA:loose coupling, distributed functionality. Agile and framework development can work together, but it takes experienced programmers and an aspect oriented approach. Segment the framework team and have the team(s) developing services continue with the design, code test of their portions. When the framework comes on-line, the service teams (already dispositioned to respond to change) should integrate their functionality with the assistance of the framework team and continue with their development. Again, this could be disastrous with junior software engineers, but with seasoned pros, it worked like magic.

Unknown said...
This comment has been removed by the author.
Unknown said...
This comment has been removed by the author.
Unknown said...

I have been reading your blog and thoroughly enjoyed doing so.

On this occasion I have to respectfully disagree. Agile for me is adapting to change quickly and efficiently. Agile from implementations perspective is the ability to manage (refactor) the codebase quickly and efficiently but from SOA point of view its ability to manage services efficiently.

In the programming world there are IDE's and other tools that help refactoring codebase a breeze where as in SOA world its the process that's put in place that makes it agile.

The primary reason for governance is agility. Ability to find, create, manage, deploy services efficiently thus reducing TTM and turn around time.

Anonymous said...

Its an awesome blog for ..
architects in bangalore as modern architectural firm working asinterior designers in bangalore know more about Green eco architectural designs are reusable materials, green designs etc.. ..… Create an Eco friendly Green design… Save Earth.. interior designers in Bangalore as of natural materials interior designers in Bangalore with almost modern concepts architects in bangalore eco friendly architectural solutions for architects in bangalore Carbon Footprint is a measure of the impact human activities have on the environment in terms of the amount of greenhouse gases produced, measured Architects in bangalore so save earth for a better future..

okeefe consulting said...

I guess I couldn't disagree more. With respect, it sounds like you may have never worked with a true Agile project. I've been involved with several "alleged" agile approaches, but when a project truly follows the principles in the Agile Manifesto, each of the arguments you made are just not valid.

Of course planning is required as it is with any software project that has multiple components. Object-oriented design is not that dissimilar to SOA, and both work well with agile principles.

Agile is not easy. It is a discipline that pays dividends when done well. It can demolish the typical waterfall approach and has been very effective in SO projects.

Agile and governance can (and must) go hand in hand. Given a proper management (scrum masters that know their stuff), Agile and SOA deliver on the promise.

Keith said...

Good post Leo. You make some good points about where agile and SOA have the greatest potential to create issues in the SD lifecycle.

But I also see the points of Daeron and John, who recommend experience teams (always helps!) and good project management and SOA governance. And to Dix's point: if you follow any methodology blindly or too formally, you're most likely in for a rude surprise. In this case, it's wouldn't make sense, regardless of the methodology, to plunge into a SOA project or platform without planning for the inevitable changes that are always requested.

Specifically, the team should understand SOA contract compatibility (so that contracts can be extended w/o breaking backwards-compatibility); should place more emphasis on unit test coverage (so that updates break fewer existing clients); and should take time to understand strategically where the business is going, so that the team can design-forward as much as possible.

-- Keith

Offshore software development India said...

Nice post and very helpful for fresh programmer. Article describes the best tips for software development. Please continue writing....

Regards:-Offshore software development company

Unknown said...

My cousin recommended this blog and she was totally right keep up the fantastic work!
Agile Software Development

Anonymous said...

Many corporations across the globe area unit turning to offshore software development. The trend started off once the massive corporations
from the west yearning for a less expensive various began to source experience from countries that offered cheaper resources.
intégrateur et expertise développement offshore sur Lyon

intégrateur et expertise développement offshore sur Lyon