Using the Analytic Hierarchy Process to Improve Enterprise
Step 3: Project Portfolio Evaluation
The AHP has a distinct advantage over other measurement methodologies because it provides decision makers with an ability to derive accurate ratio-scale priorities for both quantitative and qual- itative factors through the paired comparison process. The AHP also allows decision makers to measure the inconsistency of their judg- ments. For example, if one said that A is greater than B and B greater than C, then A logically should be greater than C. However, human beings are not always logical. The inconsistency measure of the AHP can help decision makers to better understand the relia- bility and validity of their judgments, yet it does not require them to be perfectly consistent. Indeed, one could be perfectly, consistently wrong in one’s decisions! Each decision maker must agree to the pri- orities they have established and to the process used to reach them (Figure 4.3-4).
more accurate than rating scales, due to the redundancy of mea- surement, it can become cumbersome when more than nine alter- natives are considered due to the large number of possible pairs that would be necessary to measure.) Using wireless keypads, an Inter- net portal, or paper-based surveys, decision makers typically mea- sure individual project performance using a rating scale such as the one in Figure 4.3-5. The scale should be developed with the stake- holder team, and its priorities should be produced using pairwise comparison judgments to ensure the intensities are ratio scale and meaningful to those using it.
Aside from measuring qualitative judgment, the AHP also lev- erages quantitative data. Financial objectives like return on invest- ment (ROI) and net present value (NPV), or specific quantitative metrics on sales figures or performance, can be measured using util- ity curves or step functions.
Once the team has completed its judgments and ratings, the PMO calculates the results to review and confirm the project prior- ities. This important process, known as synthesis, provides critical insight into the collaborative communication of priorities the port- folio team produced. Figure 4.3-6 provides an example of the types of views a team might receive once synthesizing the input.
The team then conducts sensitivity analysis to validate that the priorities are reasonable and to consider alternative scenarios. Sen- sitivity analysis is a powerful process that allows decision makers to ask what-if types of questions. It is valid only because the priorities
166 PROJECT PORTFOLIO MANAGEMENT
Intensity Name Excellent Very Good Good Marginal Poor None
Priority 1.000
.722 .442 .323 .104 .000
FIGURE4.3-5 A Project Rating Scale
Levine.c04.3 6/9/05 11:48 AM Page 166
produced in the AHP are ratio scale, making them proportionate.
Thus, if the team decides to consider an alternative scenario in which an objective is more heavily weighted than originally de- rived, they can simply adjust the priorities of the objectives to see how the alternative’s priorities compensate (see Figure 4.3-7).
Thus, the PMO leverages the AHP methodology to facilitate communication from the top down (leadership communicating strategic objective priorities to staff) and from the bottom up (staff communicating project priorities to leadership), synthesizing the re- sults to reach acceptable priorities to all members of the team.
Determining accurate project priorities (benefits) is essential to effective allocation of resources. Without accurate priority numbers for the projects, the process of allocating resources is futile because the presumed project benefits will be misleading. The project pri- orities of an AHP-based prioritization process reflect integrated benefit numbers and are thus more accurate than typical measure- ment approaches. That is, they incorporate the leadership’s values, experts’ judgments, and financial and other quantitative metrics.
The more highly a project is rated, the more tightly it aligns with organizational objectives, making it more beneficial.
USING THE ANALYTIC HIERARCHY PROCESS 167
Combined instance – Synthesis with respect to:
Goal: Optimize IT Portfolio To Improve Corporate Performance
Plumtree Corporate Portal Thin Client Implementation Oracle 9i Upgrade Customer Service Call Center ProServe System Upgrade Cisco Routers
Relo App Interface to PeopleSoft Iron Mountain Backup Service Sales Force Laptops Firewall and Antivirus Licenses EMC Symmetrix
Desktop Replacements PeopleSoft Upgrade SRDF Site/Service Mobile Workforce Pocket PCs Laptop Replacements
.102 .093 .071 .070 .067 .066 .064 .062 .060 .059 .056 .053 .052 .049 .040 .035
FIGURE4.3-6 Example of Project Priorities
Levine.c04.3 6/9/05 11:48 AM Page 167
Traditional portfolio management methodologies in many cases use outdated prioritization methodologies. One such approach re- quires that evaluators use measurement scales that range from 1 to 5 or from 0 to 10 to score projects. Although this methodology is typical of many gate-based approaches, it suffers from the inappro- priate use of numbers. The noted scales, 1 to 5 and 0 to 10, are most frequently interpreted as ordinals, or at best as intervals, and con- vey no information about the proportionality of the judgments being made. Moreover, it is inappropriate to perform any mathe- matical calculations on ordinal measures, even addition, because these numbers are not meant for calculation. (An ordinal scale is a set of numbers that is invariant under monotone increasing trans- formations. No mathematical operations can meaningfully be ap- plied to ordinal measures.) Thus, use of these scales will produce results that are at best approximations and at worst misleading.
Ratio scale measures are necessary to determine the contributions of a project to organizational objectives and to allocate resources meaningfully. AHP, through its ability to facilitate redundant pair- wise relativecomparisons, overcomes the shortcomings of the 1 to 5
168 PROJECT PORTFOLIO MANAGEMENT
FIGURE4.3-7 Example of a Dynamic Sensitivity Screen
41.8% Align with Corporate Strategy 10.2% Plumtree Corporate Portal 9.3% Thin Client Implementation 7.1% Oracle 9i Upgrade 7.0% Customer Service Call Center 6.7% ProServe System Upgrade 6.6% Cisco Routers
6.4% Relo App Interface to PeopleSoft 6.2% Iron Mountain Backup Service 6.0% Sales Force Laptops 5.9% Firewall and Antivirus Licenses 5.6% EMC Symmetrix
5.3% Desktop Replacements 5.2% PeopleSoft Upgrade 4.9% SRDF Site/Service
4.0% Mobile Workforce Pocket PCs 3.5% Laptop Replacements 22.7% Improve Financial Performance
16.2% Maximize Organizational Efficiency 13.0% Minimize Risks
6.3% Align with Architecture
Levine.c04.3 6/9/05 11:48 AM Page 168
and 0 to 10 scales and allows decision makers to derive ratio scale priorities. Psychologists have long known that humans are much more capable of making relative than absolute judgments. The use of redundancy in the pairwise measurement process permits better accuracy and allows measurement of the consistency of a decision maker’s judgments, an important point overlooked in traditional measurement systems. An optimal alignment of projects and allo- cation of resources requires both ratio scale measures of benefit as well as a methodology to select the combination of projects that maximizes the total benefit while adhering to constraints.
Step 4: Project Portfolio Optimization and Balancing