Death of Custom Software Development

(Disclaimer: This post is largely based on an excerpt from my contribution to the upcoming book from Thomas Erl's Service-Oriented Computing series, Next Generation SOA: A Real-World Guide to Modern Service-Oriented Computing.)

Until very recently, the typical approach for many corporate IT departments when dealing with a business problem has been – code first, ask questions later.  Many enterprise systems have been custom coded in the past.  But this trend is changing.  Companies realize that biggest value is gained from leveraging existing vendor or cloud-based solutions and integrating them into useful business solutions. 

The transformation from a development to integration mentality moves companies up the maturity curve significantly.  Agility is gained by purchasing or leasing the necessary foundational capabilities and customizing them to meet business needs.  IT no longer spends majority of its time building solutions, finding bugs, and deciding which SDLC is better.  Instead, IT teams can concentrate on delivering business value.

Organizations that adopted the integration first mentality are also forced to determine their competitive advantage strategies.  Michael Porter identified three types of business capabilities that should be considered in designing a business strategy.  They are base, competitive and differentiated capabilities.  Base capability is one that is foundational to the business, without which a business cannot operate.  This includes accounting, finance, HR, etc. as well as basic customer facing capabilities that are taken for granted.  Competitive capabilities are those that allow companies to remain competitive with each other.  For example, banks typically match each other’s lending products, interest rates, depository vehicles, etc.  Differentiated capabilities allow companies to stand out and separate themselves from each other.  In banking, examples of this include bill pay features, innovative product offerings, customer service, etc.  Michael Porter argued that companies want to be as good and efficient as its competitors in the base and competitive capabilities, but they want to excel in the differentiated capabilities to win.  Investment strategies should also match this philosophy.  Minimal necessary investments should be made in base and competitive capabilities that allow businesses to remain on par with others, but maximum possible investment should be made into increasing the degree of differentiation.

IT investment strategies should match those of its business partners.  Therefore, IT should consider which capabilities its investments support.  Just as with the business side, IT should minimize its spending on base and competitive capabilities and maximize spending to support differentiation.  The coding and integration strategies are also impacted by these decisions.  While base and competitive capabilities should almost never be custom coded, differentiated capabilities can, and oftentimes, should be.  In the modern world, however, custom development most often takes shape of configuration or scripting, as in business process design, business rules development, and vendor package configurations.  Little room is left to coding new functionality from scratch.

Service-orientation becomes a significant factor when companies adopt an integration mindset.  It becomes the central theme of everything that happens within an enterprise.  IT assets become a loosely coupled collection of vendor products, SaaS, services, mashups, and legacy applications.  Service-oriented middleware and SOA governance become key players in every project, initiative, and program.

Finally, the company’s entire culture must change.  Everyone, from business and IT leadership to sales people and developers, must embrace the integration first mentality. Without it, there will constant infighting between the development and integration clans.  Philosophical battles will ensue.  But if clear direction is given to the organization and business and IT stand shoulder to shoulder, there will be very little choice but to accept the change.

One CIO I worked for issued a very clear and concise statement of direction to his IT organization related to software development decision criteria: "Reuse before buy before build."  Everyone understood and embraced it.  The IT organization understood what decisions took priority over others.  Business stakeholders started asking questions like "Are there software assets we can reuse for this effort?" and "What vendor solutions exist for this problem?"  The entire organization adopted the integration first mentality as a result of the CIO's guidance.

While software development will never completely disappear, most IT organizations supporting mature businesses need to shift towards the integration first culture.  Let's face it, IT groups are not in the business of software development -- they are in business of supporting their business.  IT needs to concentrate on delivering business value, not building custom software!

The Architect Manifesto

IT architecture is a well defined profession. It was in existence ever since computers were first created in the 1940s and software needed to be developed for them. While it became more formalized only in the 1980s, the profession took huge steps forward in the last 30 years. Today, the world of IT architecture includes such disciplines as SOA, integration architecture, system architecture, data architecture, security architecture, etc. Yet, despite all of its different facets, IT architecture core practices, goals, and approaches remain the same across all of its disciplines.

There are several key principles that help us become successful as IT architects. No matter what problems we face, no matter what types of solution we need to apply, no matter what technologies we use, these general guidelines should always be followed. This forms the basis for the Architect Manifesto.

My research indicates that there have been multiple attempts at defining an Architect Manifesto. None of them seem to have succeeded or gained traction. Yet, I am certain that we, as architects, need the guiding light of a set of principles that will guide us in everything we do. We already unconsciously understand and follow them, but a formal definition will strengthen and inspire us. This is my attempt at verbalizing what we already know and apply in everyday work – the Architect Manifesto.

We, IT architects, guide the design and evolution of computer systems. We strive to achieve maximum value for our business partners through this work. We constantly discover better ways to architect systems. As a result, we have come to value: Simplicity over complexity Pragmatism over perfectionism Thinking out of the box over applying the same approaches to every problem Developing technology agnostic solutions over making technology driven decisions Delivering solutions to the business problems over concentrating purely on technology deliverables Looking at the big picture over getting too deep into details Long term, strategic thinking over short term, tactical thinking That is, while there is a place for the tactics on the right, we should strive towards approaches on the left.

I would appreciate feedback from all of the architects out there to make this manifesto more complete and relevant.

Business Process Excellence for Financial Services 2011 Conference

I will be presenting two sessions at the IQPC Business Process Excellence for Financial Services 2011 conference. The first one is a pre-conference workshop on Service Oriented Architecture And Business Process Management: A Comprehensive Approach To Business Performance. The second one is the first session of Day 1, Embracing Truly Integrated BPM: A Holistic Approach. This should be very exciting!

SOA & Cloud Symposium Podcast

SOA and Cloud Symposium podcast published. Listen to it at http://soasymposium.com/podcast2011.php. This podcast is part of the International SOA and Cloud Symposium Conference Series and the upcoming SOA Governance (Prentice Hall Service-Oriented Computing Series from Thomas Erl) book launch.

Architectural Thinking

I recently had to give an Architectural Thinking course. As I was putting the materials together, I wondered – what really makes a good architect? What is the difference? Is it personality, totality of experiences, different wiring, or natural inclination?

I always believed that while good architecture skills can be taught, it is the way we think that differentiates us and makes us good architects. Our natural ability to understand the problem and determine the right solution is the magic formula. This – and this alone – distinguishes good architects in our midst. You don't have to have "architect" in your job title or even be in IT to be a good architect. You can understand the technology in minute detail, know all the design patterns, possess tremendous depth of experience, have in-depth knowledge of various design techniques, etc. All this can be taught or acquired. At the end of the day, however, it is your approach to problem solving and the way you think that will make you stand out among your peer architects.

Why do I put such a heavy emphasis on the thinking style? To help you understand, we need to take a step back and define "architecture". According to Wikipedia, "the software architecture of a program or computing system is the structure or structures of the system, which comprise software components, the externally visible properties of those components, and the relationships between them." Many books, articles, and white papers provide a definition of architecture as well. However, in my opinion, these definitions are too complex and do not truly reflect the nature of architecture. My personal definition is very simple: architecture is a high level view of a technical solution to a business problem.

What does the definition of architecture have to do with our approach and thinking style? Everything! Taking all the specifics of how we actually deliver architectural solutions out of conversation, to do our job well we simply need to determine the right technical solution to the right business problem. We need to consider all the possibilities when addressing a business problem – what is there today, what is missing, what problem are we really trying to solve, is the current process the most efficient, what would deliver the most value, are we solving the right problem?.. We should be able to look at the problem from a very high level, abstract ourselves from the underlying technology or processes, and envision the most effective and efficient solution. As in the old adage, we should see the forest for the trees. Once the "ideal" solution is found, we can start concentrating on its details and understanding technology implications. Many times, we will discover that our vision cannot be readily implemented due to technology limitations, insufficient process maturity, and a host of other factors. Do not despair, however. Your vision becomes the solution goal state, and you would need to create a roadmap to get there.

Undoubtedly, some of you will disagree with the approach presented above. I can hear the arguments now – "You must consider the current situation to determine the right solution to the problem", "The project has a limited scope, and thinking broader is impractical", "The technology landscape should be considered upfront to design the most effective solution", etc. However, if we are to find the right technical solution, we must shed the old baggage. It is very hard to find new and innovative solutions if you cannot think outside the current box. To a large extent, the architectural thinking principles are grounded in the definition of what makes a good architect.

Architecture is an art, not a science. Therefore, a good architect is more of an artist – creative, imaginative, someone who can paint with a big brush. In my opinion, the characteristics and approaches listed below are the true differentiating factors among architects.

• Abstract thinking – this is the #1 quality of a good architect. You must be able to see the big picture and understand it abstractly, absent of many details that can cloud your judgment.


• Out-of-the-box thinking – many situations require us to be creative and innovative in our approaches. Good architects should be able to adapt to new situations easily and come up with the right solutions regardless of the situation.


• Clarity of vision – you, as an architect, should be able to clearly envision the solution and all of its implications including business process, technology, low level design, development, and potential phased delivery.


• Strength of convictions – architects should always try to do things right the first time, oppose inappropriate or wrong decisions, and stand up for what they believe are the right architectural solutions.


• Critical thinking – architects should always cast a critical eye towards their domain. You should challenge everything. Nothing should remain status quo or off limits. There are always opportunities for improvement. Don’t miss them because you feel comfortable with the current situation or are used to doing things a certain way.


• Problem solving skills – architects are problem solvers. Good architects strive to solve problems in the minimalist way, i.e. reaching the right solution in the most efficient manner. Even better architects ensure that they are solving the right business problem.


• Soft skills – this one is obvious. Good architects should have excellent soft skills to work well with the diverse audiences they are exposed to every day.

As in the everlasting nature vs. nurture debate, I believe good architects cannot be made – a large portion of what makes architects stand out is ingrained in how we think, act, and approach problems. To be truly effective, we should practice all the elements of architectural thinking and exhibit all the traits of a good architect.

Microsoft Azure

When I first heard about Microsoft Azure, I thought it was yet another feeble attempt by Microsoft to get in on the hype with a subpar product just to get the foot in the door. I have to admit, though, that I was pleasantly surprised. Shockingly, Azure is a robust, well designed cloud platform that may yet prove to be better than some of its competitors.

In a nutshell, Azure encompasses three products.

• Windows Azure
• Compute: Virtualized compute based on Windows Server
• Storage: Durable, scalable, & available storage
• Management: Automated, management of the service
• SQL Azure
• Database: Relational processing for structured/unstructured data
• .Net Services
• Service Bus: General purpose application bus
• Access Control: Rules-driven, claims-based access control

Essentially, Azure provides the complete cloud computing stack that allows developers to write their own applications on top of it. The self administration interface is simple and intuitive. Depending on the services you are using, it allows you to allocate your server or database capacity, hook in the service bus, and configure your application in minutes.

The Windows Azure platform introduces the Web and Worker roles. This is the implementation of a similar pattern used in WCF that decouples the network transport from the component logic. The Web role allows the applications to accept incoming requests via a variety of protocols supported by IIS. The Worker role cannot accept any direct requests from the Internet but instead can receive messages from an internal Azure queue hosted by SQL Azure. Under the covers, Web and Worker roles run in their own instances of Microsoft VM engine. All the queues and communication protocols can be configured via the control panel.

SQL Azure is no less impressive. It allows you to store data directly in the cloud in three different forms:

• Blobs
• Tables
• Relational

All of these operations are exposed through RESTful services and are really easy to use. For relational data, complete databases can be hosted in the cloud and applications can access them directly whether they themselves are hosted in the cloud or a private datacenter.

The .Net Services platform provides a couple of services – access control and message routing. Access control serves the identity validation, transformation, and federation purposes. This is all based on the rules defined through the control panel. The service bus part of the platform does what you would expect any ESB to do – service endpoint registration and access, message transformation and routing, and improved security.

Even though Azure is still a relatively immature platform, it holds a lot of promise. Microsoft has finally hit the mark. Some risks still need to be addressed, however. The typical cloud computing concerns remain – security, privacy, longevity, etc. Additionally, a platform like Azure may cause some issues for IT departments that need to adhere to regulations like Sarbanes Oxley, SAS 70, and others. Division of responsibilities, following IT governance processes, quality control, and other sticky situations may keep CIOs and other IT managers up at night. These things will eventually work themselves out through maturing the Azure platform or enhancing the IT processes. Despite the drawbacks, I believe Azure is a viable and solid platform for “cloudizing” your applications.

Cloud Computing and the Reality

Cloud computing is all the hype nowadays. All you hear from vendors, analysts, and consulting companies is that cloud computing will solve all of your IT problems. Here are just a few promises associated with cloud computing:

• Eliminate your data center
• Solve all of your scalability and on-demand computing challenges
• Simplify infrastructure
• Reduce IT spend
• Make IT operations more efficient

Are they all true? It’s a possibility. To determine what cloud computing may mean to you, examine how it fits into your IT strategy and the way you deliver technology services to the business. Here are a few things to consider.

First of all, everyone needs to understand what cloud computing really is. According to Wikipedia, “Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet”. (See http://en.wikipedia.org/wiki/Cloud_computing.) Too many people, however, forget that it is only a style and begin to associate cloud computing with specific product offerings such as the Amazon Elastic Compute Cloud (Amazon EC2), Google Apps, Microsoft Azure, and others. Companies are not limited to just third party solutions. They can implement their own private clouds if they choose.

Secondly, you need to understand the vision behind cloud computing. The idea is simple – to seamlessly provide flexible, on-demand computing resources whenever necessary. This is not a revolutionary development. The Application Service Provider (ASP) model has been in existence for years. Infrastructure outsourcing practices have been utilized a long time before cloud computing became a term. So, what is all the hype then, you ask. They keys are the ubiquitous nature of the protocols used, increased reliability of the Internet, and the packaging of the offering as a generic service. Cloud computing, as a general approach, may support outsourcing of specific applications, generic computing resources or platforms, and software services. It may potentially lead to outsourcing of the whole data center.

Finally, all the pros and cons behind cloud computing need to be considered. Having someone take care of all your computing resources without investing into expensive data centers is an appealing concept but loss of control and unreliable SLAs may be a cause of concern for a number of businesses. Since the Internet is the primary communication mechanism for the public cloud, its reliability and performance need to be questioned whenever considering third party cloud offerings. Private clouds provide better control, reliability, and performance but what is the real difference between those and existing data centers? In my opinion, aside from following a different architectural model of allocating computing resources, nothing. On-demand computing is a great concept but making it work effectively is a tough task. Technologies exist today to dynamically divert unused resources to those applications that need them most. Grid computing, virtualization platforms, and others all provide these capabilities. However, there are limitations. Whenever maximum capacity is reached, hardware needs to be added. No software trick will work to cover this up. Therefore, efficient capacity and pipeline management need to exist to make cloud computing an effective and viable platform.

While there are some cloud computing zealots (http://www.infoworld.com/d/architecture/soa-realized-enterprise-computing-cloud-computing-146) and realists (http://www.gcn.com/Articles/2009/03/09/Guest-commentary-SOA-cloud.aspx), many are still cautious about this technology. And for a good reason. In my opinion, cloud computing has proven its worth in a number of situations but it is still not ready for the enterprise. Public clouds are too fickle for really demanding applications. Private clouds have not yet been effectively built. More importantly, however, lack of cloud computing standards and consensus among the key players will present challenges for anyone trying to enter this arena.