Analysis of Alternatives

Avoiding Bias and Ensuring Confident Decision-Making

An effective Analysis of Alternatives (AoA) can be a lengthy process. Still, the return on investment can make it a powerful tool for any organization. It can help teams avoid delays, stay on budget, and avoid costly project direction changes. A successful AoA will yield a well-researched, thoroughly scoured, and evidence-laden decision matrix for confident and defensible decisions. As AoAs are a required part of doing business in the Department of Defense and Federal Civilian agencies, many agencies have produced their own guidelines and procedures. Therefore, effectively executing the AoA process is more than nice to do – it’s necessary.

Influenced by best practices in industry and government, and our work supporting federal customers, SkyePoint Decisions, Inc. (SkyePoint) has put together the SkyePoint Alternatives Decision Framework. The framework includes six areas for an AoA team to focus its efforts on to produce the most valuable outcomes possible:

Following is an overview of these steps.

Define Parameters

Before analysis can begin, some ground rules need to be established. Without a solid foundation, there is a real chance of bias sneaking into the results. Having the following guides and bumpers as the team goes through the process will save time and money down the road.

The parameters are:

• Define a need for a recognized gap between the status quo, the baseline for comparison, and the future requirements. Again, this is most likely tied to a mission or business goal, so that relationship should be documented for framing the entire decision analysis process.
• Define the functional requirements of this Need so that each alternative can be consistently compared for effectiveness and viability.
• Decide and agree upon a manageable and realistic timeframe for the AoA process with a commitment to participate from start to end.
• Include a wide variety of team member skills to cover all facets of the analysis to produce a gapless and error-free result.
• Determine the selection criteria used to choose the best alternative in advance to avoid bias. Weight the criteria by relative importance as compared to the identified Need.
• Create a written plan to guide the AoA team through the upcoming rigorous stages of Discovery, Documentation, Dissection, Decision, and Due Diligence. Publish the plan to the group, make it readily available, and update it promptly, as needed. The fewer updates required, the smoother the process.

Some critical components of this AoA plan are:

• Methodologies to identify alternatives and criteria that will be used to determine feasibility in the Discovery stage
• Critical questions to answer for each alternative, the basis for any estimates that might need to be made, and a predetermined list of measures by which to rank each option, all to be used in the Dissection stage
• The selection criteria to be used in the Decision stage

Discover Alternatives

Next, find some alternatives to compare. To avoid bias due to familiarity, comfort, or sentimentality, don’t start with the status quo. Instead, begin with an open outlook as the team research ways to satisfy the gap in the mission/business goals. Gather innovative alternatives from industry events, conferences, product partners, trusted technology integrators, and anywhere else to find relevant sources.

Treat this stage like brainstorming, where no idea is bad, and all are considered. The alternatives will generally fall into several categories. Knowing this can fuel thoughts to generate more options and to satisfy the Due Diligence stage later, including:

• Outsourcing the required capabilities
• Using a government off-the-shelf (GOTS) product or service
• Instituting a process or organizational change
• Implementing an IT system, whether greenfield development, a commercial off-the-shelf (COTS) product, or updating a legacy system

Select the best alternatives based on feasibility and viability using the methodologies and criteria from the AoA plan. Ideally, the top three alternatives should be chosen, though the team can decide on the number with time and effort in mind. Finally, add the status quo to the final group of alternatives to go on to the next stage.

Document Alternatives

Documenting alternatives requires a good comprehension of what is needed for the team to analyze the data, generate metrics, and identify criteria. Detailed information for each status quo and top alternatives must include such points as:

This step can be tedious and time-consuming depending on the availability and completeness of the information needed so that an apples-to-apples comparison can be made. Good intra-team communication is essential throughout, but especially in documenting the alternatives. Regular updates and sharing with the group allow a continuous review process to elevate the quality of documentation the team produces. When choosing team members, look for good collaborators who can best support this process.

Dissect Alternatives

Dissecting each alternative’s documentation to generate the agreed-upon metrics and selection criteria seems to be a simple matter of dragging in values to formulas and collecting them into an Excel table, right? Yes, but….no matter how prepared the team is, generating the metrics is hard work. A well-trained, ready, and cohesive team will have an easier time, but it’s still challenging. Even with the methodology laid out, quantifying effectiveness and qualitative benefits can feel like trying to fit a round peg into a well-defined square hole.

A good list of scores to create for each alternative includes the following:

• Documented qualitative benefits, both quantifiable and non-quantifiable
• Ranked general risks by significance to the mission/business need
• Risk-adjusted cost estimate over the alternative’s lifecycle
• Risk-adjusted benefit estimate over the alternative’s lifecycle
• Financial metrics like net present value, benefit-to-cost ratio, and payback period

Decide Best Alternative

The breakdown of each alternative into comparable metrics should yield healthy options from which the decisionmaker can choose. Of course, there is no hard and fast rule for which alternative is the winner. Still, a decision that goes against a ranked ordering by risk, qualitative and financial metrics must be defensible by sound logic and reasoning.

Due Diligence Options

Finally, we come to some options that can be performed along the way to make the Analysis of Alternatives more robust and stand against the scrutiny of quality assurance. The first due diligence opportunity is an extra level of analysis around the top alternatives, including the status quo. Particular attention can be given to exploring how one alternative’s differentiating factor could be applied to other options to produce more significant benefits. This synergy of choices should be thoroughly scrutinized throughout the AoA process to identify any possible detractors.

Lastly, due diligence can be performed by conducting a sensitivity analysis. Like the first due diligence point above, instead of looking for greater benefits, look for ways that a singular change to an alternative’s metric scores would impact the final ranking. Again, this gives a measure of sensitivity to each alternative’s ranking, which may influence the final choice.

The Alternatives Decision Framework assures the team that a very well-informed choice will be made. There is undoubtedly a need for such effort and the people and process management that make it go as smoothly as possible and drive it through to completion.

If your organization is embarking on new efforts to modernize or upgrade existing capabilities, reach out to SkyePoint Decisions to be your partner of choice to execute the Alternatives Decision Framework and achieve your mission goals.