Sensitivity Analysis in Decision Making
Sensitivity Analysis in Decision-Making · Does the conclusion hold when the assumptions change?
Does the best alternative stay the same when the weights or scores change? Decision analysis evaluates several alternatives and calculates scores to set priorities, but the result is not always definitive. A small change in the importance of a criterion, or a different expectation about the future, can flip the ranking. When choosing a laptop, A may come out ahead if you care most about performance, and B if you care more about price. What matters is not just confirming the current winner but understanding under what conditions the choice changes.
Sensitivity analysis is a way to check how much the result changes as you vary the assumptions and inputs used in the decision. This lesson systematically brings together what we did piecemeal in Chs. 01, 02, 04 and 06.
Why sensitivity analysis is needed
Which matters more, price or performance?
What score do you give each product's performance or quality?
How likely is a new product launch to succeed?
How much could the investment cost or the expected return vary?
In reality such values are hard to pin down perfectly. If a result calculated from a single value flips easily with a small change, that decision deserves careful review; if the same alternative keeps coming out ahead when you change the inputs within a reasonable range, the choice can be considered relatively stable.
Ranking analysis under changing weights
Suppose three laptops are rated on two criteria, price and performance. Both scores are out of 100, and the price score is higher the cheaper the laptop.
| Alternative | Price score | Performance score |
|---|---|---|
| Laptop A | 70 | 95 |
| Laptop B | 90 | 80 |
| Laptop C | 80 | 85 |
w: price weight · Pi: price score · Ti: performance score
| Alternative | Calculation (w = 0.4) | Overall score |
|---|---|---|
| A | 70(0.4) + 95(0.6) | 85 |
| B | 90(0.4) + 80(0.6) | 84 |
| C | 80(0.4) + 85(0.6) | 83 |
At a price weight of 0.4, A scores highest, but if price is rated as more important, the result can change.
Finding the point where the ranking flips
With a price weight below about 42.9%, A comes out ahead; above it, B. However, finding where the overall winner changes also requires checking the other alternatives' scores. In Figure 1, C ties with B at w = 1/3, but A is still higher there, so with these data C never ranks first at any weight. Finding the threshold at which the result changes in this way lets you explain clearly which alternative comes out ahead over a given range of weights. The lesson's assumption of 0.4 is close to the threshold of 0.429, so this conclusion is fairly sensitive to the price weight.
Comparing results across scenarios
Besides changing inputs one at a time, you can also compare situations in which several assumptions change together. This is called scenario analysis. Let's vary the probability of success in the new product launch problem from Ch. 02.
p: probability of success · G: profit if successful ($200,000) · L: loss if it fails ($80,000)
| Scenario | Probability of success | Calculation | Expected value |
|---|---|---|---|
| Optimistic | 80% | 0.8 × $200,000 − 0.2 × $80,000 | $144,000 |
| Base | 60% | 0.6 × $200,000 − 0.4 × $80,000 | $88,000 |
| Pessimistic | 40% | 0.4 × $200,000 − 0.6 × $80,000 | $32,000 |
Even when the probability of success falls, the expected value stays positive. That alone, however, does not prove that launching is a good decision. Factors such as the investment, other options and the risk of running short of funds must also be considered, and the expected value is only a probability-weighted average; it does not guarantee the return you will actually get (Ch. 03).
One-way and multi-way analysis
One-way sensitivity analysis
Change only one input at a time and hold the rest fixed. Its advantage is that it makes the key drivers of the result easy to identify, and it is summarized in a tornado diagram, which stacks bars showing the range of influence of each input.
In Figure 3, the input with the largest effect on the expected value is the probability of success p (range $32,000 to $144,000). Next come the profit if successful, G, and then the loss if it fails, L. This means that if you spend money on collecting more data, spending it on estimating the probability of success more precisely is the most effective use. Since the bar lengths depend on how the range of each input was set, you should also state the basis for those ranges.
Multi-way sensitivity analysis
Change two or more inputs at the same time. If you vary the price weight and the performance score together, you can see which alternative is chosen for each combination of the two conditions.
Figure 4 is a map of the winner when the price weight and laptop A's performance score change at the same time. The lower A's performance score is rated below 95, the smaller the region where A wins, and below about 90 a region also appears where C wins at moderate price weights. This means that C, which never ranked first when only the weights changed, can rank first when the scores change too. However, when several values change at once it becomes hard to tell which change drove the result, so set the ranges of change to fit the purpose of the analysis and state the assumptions of each scenario.
Probabilistic sensitivity analysis
Instead of moving the inputs only to the ends of their ranges, you can also give each input a probability distribution, draw from them at random many times, and look at the distribution of the result.
In Figure 5, when the uncertainty in the probability of success, the profit and the loss are all included, the mean expected value is $87,740, and in 90% of cases it lies between $28,660 and $147,260. The expected value is negative in 0.6% of cases, so under these assumptions the conclusion that launching is favorable is very stable.
How to interpret sensitivity analysis results
| Result | Meaning | Use in decision making |
|---|---|---|
| Stays first over a wide range | The result is relatively stable | Review the reliability of the current choice |
| Ranking flips with small changes | Sensitive to specific assumptions | Re-examine the weights and evaluation data |
| First only under specific conditions | A conditionally favorable alternative | Check whether those conditions are realistic |
| Results differ widely across scenarios | High uncertainty | Consider collecting more data or managing the risk |
Being stable does not mean being best in every situation. If there are factors left out of the analysis or assumptions set incorrectly, sensitivity analysis alone cannot fix that. So it is good practice to report which values you varied, how you set the ranges of change, and under what conditions the ranking changed.
Links to other decision-making techniques
| Technique | What sensitivity analysis targets |
|---|---|
| Decision trees (Ch. 02) | Check whether the best choice changes when the probability of success or the costs change |
| Expected utility (Ch. 03) | Check how assumptions about risk preference or the utility function affect the choice |
| MCDM · AHP (Chs. 04 · 05) | Vary the criterion weights to check the alternatives' priorities |
| TOPSIS (Ch. 06) | Check closeness and rankings as the weights, scores or set of compared alternatives change |
In particular, in AHP, the fact that the consistency ratio (CR) is acceptable and the fact that the ranking is stable in a sensitivity analysis mean different things. The consistency ratio assesses the logical consistency of the pairwise comparison judgments, while sensitivity analysis assesses how stable the result is to changes in the inputs.
In a weighted sum with price and performance weights of 0.4 : 0.6, laptop A ranked first with 85 points, but when the price weight exceeded 0.429, B ranked first. The expected value of the new product launch ranged from $32,000 to $144,000 over success probabilities of 40–80%, all positive, and in 20,000 simulations that included the uncertainty in the probability of success, profit and loss, the expected value was negative 0.6% of the time.
What matters more than the current winner is the conditions under which the winner changes.
- Sensitivity analysis: Check how changes in inputs such as weights, probabilities and scores affect the result.
- Threshold: Finding the point where the result flips lets you state concretely the conditions under which the choice changes.
- Methods: Choose one-way (tornado), multi-way (combination map), scenario or probabilistic analysis to suit the purpose.
- Interpretation: A stable result is not necessarily the right decision; the adequacy of the input data and the model must also be reviewed.
- Reporting: Present what you changed, over what range, and where the ranking changed.
Wrapping up the seven decision analysis topics
| Research question | Main methods | Lesson |
|---|---|---|
| What do we compare alternatives on? | Goal · alternatives · criteria · weighted sum | 01 |
| How do we compare uncertain outcomes? | Decision trees, expected value, value of information | 02 |
| How do we reflect attitudes toward risk? | Expected utility, certainty equivalent | 03 |
| How do we combine several criteria? | Multi-criteria decision making (MCDM), weighted sum | 04 |
| How do we set the importance of the criteria? | AHP pairwise comparison, consistency ratio | 05 |
| How do we rank the alternatives? | TOPSIS, relative closeness | 06 |
| How robust is the conclusion to the assumptions? | Threshold, tornado, scenarios, probabilistic sensitivity | 07 |
In decision analysis we covered, in turn, what you want (goals and criteria), what is uncertain (probabilities and outcomes), and how robust the judgment is (sensitivity). Whichever method you use, the key is not to produce a right answer from a single number but to state your assumptions and rationale and report the conditions under which the conclusion changes.
Next is 15 Optimization · 01 Understanding optimization. We will look at how to allocate limited resources and how to find the best solution within constraints.