"How do we manage all the requirement we have and how do we get people to commit to something when everyone has a different perspective based on their job role, and nobody wants to commit to a final decision for fear they will not be able to modify it?"

"How do we give people the freedom to change their minds, yet deliver concrete estimates on time and cost to our managers?"

"How much will this cost us and how can we estimate it knowing we will likely change our minds during the project based on previous projects, and how can we get early and consistent involvement in the development process?"

"How can we ensure that necessary quality is met?"

And many more similar concerns are typically expressed.

And for vendors or internal IT tasked with building a system (or customizing an existing system) the concerns are:

"How can we ensure profitability on our work when we know change will occur? (for internal IT they may be managed and accountable for profitability if they are managed like a profit center). If not profitability then "How can we possibly commit to a date and cost when we know this will be a moving target as we evolve the solution?"

"We have an abysmal record of missed deadlines, cost over-runs, quality problems and our users not being happy with our delivered systems. What can we do to improve this?"

"How do we manage the scope changes and ‘feature creep' placed on us by the business people/stakeholders? This is what typically destroys our profits and/or time commitments and cost commitments, often resulting is a sever quality diminishment. How can we give people what they want but manage this change?"

Any assigned domain expert should have the reach to get answers to questions quickly outside their domain knowledge when required. For a non-trivial project it will likely cover many domains and it may not be possible for that individual to have a mastery of all of them, however this person should be able to expeditiously get answers from others when required, This is a critical success factor to any Agile project.

I have evolved over many years this approach to software profitability and risk management in a very well tested and disciplined way. It has evolved since 1998 when I was CTO of a niche financial services software firm, which was sold in 2004. I then started the firm agilefactor and significantly evolved these ideas. If any of this content seems like 'academia' or 'theory' I can assure you these are all proven real-world strategies, having been applied by myself and many others for years and years.

Basically when pricing a deal, and how you execute it, everything eventually comes down these moving critical dimensions...

1. Time
2. Scope/Features
3. Quality (never really up for discussion as software nearly always must be 'scalable and production quality')
4. Cost

It is CRITICAL you never let a client box you in on all of the three of these items if it can be avoided or you may have to 'walk away' from a project (many in internal IT do not have that luxury but at least this article will show you how to push back in a way that is hard to be disputed by the forces trying to box you in to possible assured failure).

Your client gets to drive only two of the three items at most (such as Time and Cost), while you driven the third (Scope in this example) or the remaining items you control (you must always have at least one that the client explicitly gives you control over).

This is critical to profitable software sales both for commercial and custom one-off development but many fail in this area in their eagerness to close deals (this is especially a problem with salespeople who are not put in check on what they can promise the client, and will promise you right into a massive loss - a lesson I learned first hand early in my career).

For example, if a client says:

"OK. We just received these parameters from our IT Committee for the project we would like you to bid on. Vendors have until October 1st to achieve these 120 individual scope items in the attached 600 page scope document we have assembled (the first sign of trouble). Vendors have $200,000 to build this system. This is non-negotiable and the software must be production quality, scaling to 500 concurrent sessions per machine with an average response time of 1 second and max 3 no more then 25% of the time."

We really have no idea if we should accept these terms however there are ways where we can accept the terms (if we feel close to OK on the scope) as we know clients always change their mind extensively and there is almost always a large delta is what is actually delivered compared to what is planned in Agile projects (which is one of the reasons they are so successful). The changes are typically critical and other development processes would have failed to properly address these changes were it not for the timeboxed iterative delivery to the clients and the client's direct involvement with the team and heavy exposure of being made ‘fully involved team members who are an actual part of the process'. This has many benefits psychologically as the stakeholders feel as if this is not 'you guys coding in dark cubicles' it is ‘our project where we work collaboratively to a successful result'.

A good response to this example case is:

"We need to understand your documented scope in detail before we can commit to our ability to achieve all 120 use cases by October 1st at that price and quality level. Anyone who does commit without a detailed assessment is guessing and the outcome we have seen in these cases is rarely positive. You may get lower bids but none will be as thoughtful as ours in terms of not making false promises and our ability to either be able to do this within your criteria or us telling you it cannot be done and leave this to another vendor who believes otherwise. We will put forth our response and you should know for fixed price engagements such as what you are asking of us, we require a change order process for deviations from your scope. As you seem confident you have captured the scope in this large scope document these change orders should not be a problem for you as you are very confident in your 600 page scope document” (as we knowingly look at each other with glances saying ‘of course there will be massive change').

Here we are playing off the client’s unfounded confidence that they can actual scope a system upfront with their large scope document. Studies have debunked that years ago and it is common knowledge that this cannot be done except for the most trivial systems. We use this to our advantage with the mandatory ‘Change Order’ process where any deviation is estimated from a time, quality and cost perspective and presented to the client. They can then decide if they really need that new feature or that significant modification of an existing one. In many cases a client will drop features in order to get the new functionality they desire in a trade-off to not cause disruption. This however is not always possible depending on the nature of the change.

Is this an unethical approach, by playing off the ignorance of the client? I do not believe it is. We can attempt to educate the client on the realities of monolithic up-front scope definition but it is dependent on the existing relationship. Here it is a competitive bid environment so if we want to engage this project we have no choice but to cover ourselves with the change order process and if we try to explain to this client that the 600 page document they just created is not worth all that much, it may cost us our ability to even bid on the project. Again, it is situational and I would always advocate educating the client if possible.

It is a ‘best practice’ to require all stakeholders to sign an acceptance document at the end of every two week iteration you present to them and allow them to work with. Sometimes the iteration will fail, and they will refuse to sign until specific changes are made, but it is far better to capture and implement these changes every two weeks that deliver a monolithic system that has potentially hundreds of issues. I highly recommend this practice to all Agile software builders, as the weight of a stakeholder signing their name to an acceptance document will facilitate them performing a far more vigorous and detailed review of the work, and will also provide a paper trail for your organization that you were on track during the software development lifecycle, and when things deviated from the stakeholders vision, you can show you corrected the issue, and there was an issuance of change orders if necessary and in general let the system moved forward to success rather then the more common occurrence of stakeholders not being allowed to ‘touch’ a system until it is complete (a nearly sure recipe for disaster and unfortunately heated finger pointing which can lead to litigation).

We go back and with out team do some heavy thinking and find that we feel good about getting 100 items done by December 1st with a high probability (85%). I will expand on the probabilistic nature of what I just described in the next section as it is the evolution of this framework.

In this case, the client agrees to pay for any change orders and suffer any time delays based on their requests (clients are almost always over-confident in their ability to document scope up-front as I mentioned, be it in a document, perhaps with a prototype and other forms of scope definition.

We can come back to the client and say:

"We believe with a high probability we can achieve all of your documented 120 use cases by Oct 1 at a cost of $400,000 and at the level of quality you require."

Here is concrete and personal example. While a leader at an innovative Software product company, we planned a new engagement with one of the world’s largest financial services organizations. We put in a rather severe change management process which saved us. The original budget was around $5,000,000 and we agreed to their requests on time based on the scope they presented within what our product could handle. But the money they spent on their change orders which required significant extension work on our product to meet their truly one-off requirements (as they kept changing their mind which in the context was necessary) was well over $5,000,000 (something like $7,000,000). We managed this without splitting our code base, instead building in an extensibility capability which would serve us well later. If we had given them the freedom here to make changes at will or over-represented what our product could do to land the business we would of lost our shirt. I see this often when a company will promise features they have not even built to a new client and underestimate the cost of making these features real. They had so many extensions they wanted us to build they just kept paying and accepting delays from the change orders we were constantly creating.

So the client drives at most two dimensions, and you must drive the third or you risk loosing money, potentially lots of it. Any changes to a 'fixed price' must be done with an incredibly rigorous 'change order' process where all changes from the plan are scoped, priced and the impact is presented to the customer so they can decide if it is worth it. I often would meet with the client described above and say ‘OK we can do this but it will cost $25,000 and delay us two weeks. What do you want to do'. They were often mad, sometimes lashing out at our product for its lack of this feature, but we never misrepresented what we was already built in and what was not. It was hard to accuse us of wrongdoing for not having a feature they wanted that we never claimed we had built. I was always able to point that out and diffuse the situation. When they thought about it, they really had no choice. They had already spend millions, and THEY were deviating from the plan they committed to with confident assurances that there would be little to no need for ‘change orders’ (this is a common occurrence in my experience with a correlation to the larger the company the most convinced they are the scope will not change. It always end the same: Massive scope change. Again, this is why Agile is the only responsible process for most projects as software is a ‘Wicked Problem’ something I define later on. Change will occur. The question is how do you handle it both from a profitability perspective and an execution perspective. Agile practices are built around expecting change.

You may think this all makes sense. However, even the concepts presented above have evolved and are now (when taken alone) considered overly simplistic based on the evolutionary learning I and many others have experienced. We have expanded this framework as described below.

Before you determine these three dimensions, you must determine the customer's 'Risk Tolerance' on failure. This is the latest evolution in vastly improving this framework. You may think 'they have no risk tolerance'. If you consider this statement, they must accept risk as It is next to impossible to have 100% success probability for future events. It infers we can predict the future (as corporations do today with project plans with no statistical component). This is one of the key reasons custom software is such a failure in the U.S. with losses well over $100,000,000 (based on Standish Group Research). It is the arrogance of upfront scope definition combined with misguided project plans and GANNT charts that have no concept of the statistical confidence of each item or concept of our inability to predict the future. IN addition it is common that there is a lack of any kind of Risk Management. People follow project plans right off a cliff rather then say ‘We know we cannot predict the future so we will take broad brush-stroke approaches to planning and do our best with our iterative development to stay on track’. When I ask companies with traditional GANNT charts what the probability/confidence is on each item (especially on critical path items) I am typically met with a blank stare. It is very hard to explain that each item needs a new column that captures the confidence it will be achieved by the specified date. You can then multiple these out and take a weighted average on the critical path items to arrive at results that show typically that there is no way the dates on the plan can be achieved (hence the common practice of project planners ‘padding’ schedules in any way they can).

You can carry out this process as described explicitly, where you explain the concepts I will define below of the Probability Curve and what I have already described, as well as Risk Management with your clients and make them a partner in the process. Another approach which unfortunately is taken by most as the maturity of organizations has not caught up to these concepts and if you are alone among others NOT following this process, you will be forced to improvise. This can be done by implicitly determining a risk tolerance based on comments from the client and you and as many others input as possible to form a judgment and assign them a risk tolerance. What is this risk tolerance I describe? Read on and I will explain.

This is the distribution curve we must embrace:

You establish the client's willingness to accept the unavoidable possibility of failure across any of the four dimensions (we cannot predict the future) and then find that point on the curve moving up from the X axis point you have for the client (the X axis is probability of success) and take the area to the left of that point.

The total area under the curve is 100% so the farthest right point would indicate 100% confidence (again basically impossible to achieve where most organizations assume 100% by default, something that must change if our industry will ever ‘grow up’. As we cannot control the future, we can only manage it and the risks, we must know how 'risky' the client wants to be (again by asking them or inferring it ourselves - often manifested in a 'public' date and an 'internal' date which is past the public date if we know the client will not cancel the project or penalize us tangibly for missing the public date).

It is a typical 'risk/reward' situation. If the customer takes on more risk we might save them money, or deliver early. If they want less risk, we must significantly increase our cumulative resources attached to a project with the knowledge that 'adding people to a late project makes it later'. This we know as fact so adding people is really only an option in the early phases. This has a large impact on early project planning.

We need to understand if a customer is OK assuming a 15% risk of failure for example (all future events have unknown outcomes except for death and taxes (grin)) or if the client is willing to as much as double (!) their budget for a say 10% increase to 95% probability of future success across all dimensions (the tail is actually much more steeply sloped then shown above, but hopefully you get the idea). A 95% success promise is incredibly risky for you the software builder, so I recommend at least a 100% increase in the total resources you have at your disposal, again knowing that more people does not always translate to a better outcome (I would take 10 amazing developers over 100 mediocre developers any day. Studies are behind me on this as well with the well established fact that the best developers are as much as 28x more productive then the worst). It quite literally is often a 100% increase in total resources allocated for a project to move from 85% to 95% even though it is a 10% gain in probability, hence the very long tail. The highest I have ever had a client go is 90% and we actually had two teams in two locations for redundancy (they didn’t know about each other). One team almost failed but delivered a solution, however the other team’s solution was superior so that is what we went with. This gave us our high probability due to the redundancy and the client paid dearly for this safety.

For example, the development team may say:

"We had a terrible two weeks. Our lead Cryptography expert was sick so we moved those features into the next iteration and we severely underestimated the complexity of the mainframe integration and User Profile Customization. Therefore we could only achieve 50% of what we had planned for this iteration. I would say from our agreed upon 85% confidence we are now closer to 75%. We can see if we make it up in the next iteration or we can take measures A, B and C to get back to 85%. What do you want to do?".

As for the client, they can also be blissfully unaware of how we manage the process of our internal development, but this well established process makes it incredibly hard to fail.

Here is an example with risk tolerance. Based on negotiations with the client they have agreed to a risk tolerance of 85% (a 15% risk of failure). Based on this, you can now estimate knowing your margin for error. The customer has scope requirements, and cost requirements but is fairly open to time.

You come back to the client and say:

"Based on an 85% confidence we feel we can deliver your scope at the cost you describe at some point between April 1 2006 and July 1 2006’". The client may protest, saying (for the first time) they were hoping for a March 1 delivery. You confer with your colleagues and respond that this would be possible at a 70% Confidence (30% chance of failure). And so the negotiations follow...But at least now you have very concrete 'dials to turn' and it is hard for the customer to argue with the fact base supporting you.

Figure 2: The Forces Driving your Ability to Achieve the Four Key Dimensions

Almost all custom software is after all (with little debate) a 'wicked problem' (to see a great definition of 'wicked problem' see http://www.cognexus.org/id42.htm or the text at the end of this article.

After reading this, you will understand why the waterfall software method has failed so miserably (and continues to cost the US economy billions in waste for companies too out of touch to fix things) unless perhaps you are building life critical systems, real-time systems, or some other specialized variant like embedded software (however even in those spaces I have read articles of success with Agile concepts and had discussed this with a life-critical systems developer using Agile with great success.

NOTE: This content is taken from the book "Agile Software Factories" by Damon Carr to be published by Addison-Wesley in 2006.

For a much more detailed discussion of these topics and what this author considers ’Required Reading’ for all software leaders is:

"Waltzing with Bears", Dorset House Publishing Company, Incorporated (March, 2003), ISBN: 0932633609

Wicked problems always occur in a social context -- the wickedness of the problem reflects the diversity among the stakeholders in the problem.

Most projects in organizations -- and virtually all technology-related projects these days -- are about wicked problems. Indeed, it is the social complexity of these problems, not their technical complexity, that overwhelms most current problem solving and project management approaches. (See graphic of wicked problem solving process below.)

Some specific aspects of problem wickedness include:

1. You don't understand the problem until you have developed a solution. Indeed, there is no definitive statement of "The Problem." The problem is ill-structured, an evolving set of interlocking issues and constraints.
2. Wicked problems have no stopping rule. Since there is no definitive "The Problem", there is also no definitive "The Solution." The problem solving process ends when you run out of resources.
3. Solutions to wicked problems are not right or wrong, simply "better," "worse," "good enough," or "not good enough."
4. Every wicked problem is essentially unique and novel. There are so many factors and conditions, all embedded in a dynamic social context, that no two wicked problems are alike, and the solutions to them will always be custom designed and fitted.
5. Every solution to a wicked problem is a "one-shot operation," every attempt has consequences. As Rittel says, "One cannot build a freeway to see how it works." This is the "Catch 22" about wicked problems: you can't learn about the problem without trying solutions, but every solution you try is expensive and has lasting unintended consequences which are likely to spawn new wicked problems.
6. Wicked problems have no given alternative solutions. There may be no solutions, or there may be a host of potential solutions that are devised, and another host that are never even thought of.

Damon Carr is the CEO and Chief Technologist of agilefactor, based in NY, New York. The company continues its growth via Microsoft.NET Leadership, Software Engineering best practices Agile Process Evolution and the new book by Mr. Carr covering "Agile Software Factories" to be published in early 2006 by Addison-Wesley. Prior to joining agilefactor, Damon was the CTO and Co-Founder of Monetaire, a global company with offices in Europe and the US, where he spearheaded all technical innovation, including their flagship product which was named a 'top three solution' in Europe by Forrester Research. Damon continues to speak at the software industry's most prestigious conferences where he is a featured speaker on 'Agile Processes and Software Factory Evolution', he also leads various high profile global user groups.

Contact the author.