What is Sensitivity Analysis

Sensitivity analysis is a method used to determine how different values of independent variables affect a particular dependent variable under a specific set of assumptions. This technique is essential for understanding and improving models, making it invaluable in decision-making processes.

The roots of sensitivity analysis can be traced back to scientific and engineering studies in the early 20th century, where it was used to assess the reliability and performance of systems under varying conditions. Over time, its application expanded into economic modelling, risk management, and policy analysis, reflecting its versatility and importance in handling complex, uncertain scenarios.

Sensitivity analysis continues to evolve with advancements in computational methods, becoming an integral part of modern analytical practices.

Join us as we explore sensitivity analysis' methodology, its benefits, and its limitations.

Definition of Sensitivity Analysis

Sensitivity analysis is a technique used to determine how different values of an independent variable affect a particular dependent variable under a given set of assumptions. This analysis is particularly useful in assessing the impact of changes in input variables on the outcome of a model or system. By systematically varying the inputs within specific ranges, analysts can identify which inputs have the most influence on the outcome and how the outcome changes in response to variations in input values.

What is Sensitivity Analysis

Sensitivity analysis investigates the impact that varying inputs have on output within a given model, helping to understand how changes in those inputs can affect the results. It's a tool often used across various fields, such as finance, engineering, and environmental studies, to test the robustness and reliability of models under different scenarios. By identifying which inputs significantly influence the output, stakeholders can make better-informed decisions, prioritise areas for further analysis or data collection, and understand the potential variability in their models. This analysis is essential for decision-making processes, especially in situations characterised by uncertainty and complexity.

How to do Sensitivity Analysis

Conducting sensitivity analysis involves several steps designed to systematically evaluate the impact of input variations on the output of a model or system. Here's a step-by-step guide to performing sensitivity analysis:

Step 1: Define the Model and Objectives

Start by clearly defining the model you're analysing and the specific objectives or questions you aim to address with the sensitivity analysis. Understand the inputs, outputs, and how they are related within the model.

Step 2: Identify Key Inputs

Select the inputs or parameters within your model that you believe might significantly affect the output. These can include variables with uncertain values or those expected to have a considerable impact on the model's results.

Step 3: Determine Ranges and Distributions

For each key input identified, establish the range of values it can take. If possible, assign probability distributions to these inputs based on historical data or expert judgment, reflecting the expected variations in these parameters.

Step 4: Choose a Method for Analysis

Select an appropriate method for conducting the sensitivity analysis. Common methods include:

One-at-a-Time (OAT): Vary one input at a time while keeping others constant to isolate its effect on the outcome.

Monte Carlo Simulation: Randomly sample input values from their distributions to assess the probability distribution of the output.

Local Sensitivity Analysis: Analyse the effect of small changes in inputs on the output.

Global Sensitivity Analysis: Evaluate the impact of varying all inputs simultaneously across their entire ranges.

Step 5: Execute the Analysis

Using the chosen method, systematically vary the inputs according to their defined ranges or distributions and observe the corresponding changes in the output. This step may involve computational models, simulations, or manual calculations depending on the complexity of the model and the method used.

Step 6: Analyse Results

Examine the results to identify which inputs have the most significant impact on the output. Look for patterns or insights that can help in understanding the model's behaviour and its sensitivities to different parameters.

Step 7: Interpret and Communicate Findings

Interpret the findings in the context of the model's objectives and the real-world scenario it represents. Prepare to communicate these insights to stakeholders, highlighting key sensitivities and implications for decision-making.

Step 8: Make Informed Decisions

Use the insights gained from the sensitivity analysis to inform decisions, policy-making, or further research. The analysis might indicate areas where more precise data collection is necessary, or it could help in prioritising resources and efforts based on the sensitivity of different inputs.

Remember, sensitivity analysis is an iterative process. As new data becomes available or as the model evolves, you may need to revisit and update the analysis to ensure the findings remain relevant and accurate.

Example of Sensitivity Analysis

Here is an example involving an IT service provider aiming to improve its incident resolution process:

Optimising Incident Resolution Time

An IT service provider is focused on reducing the average time to resolve incidents to enhance customer satisfaction and operational efficiency. The leadership team decides to conduct a sensitivity analysis to understand which factors most significantly impact resolution time and where to focus improvement efforts.

Define the Model and Objectives

The objective is to minimise the average incident resolution time. The model will consider various inputs such as the number of IT staff, their skill levels, the use of automation tools, and the complexity of incidents.

Identify Key Inputs

Key inputs identified for the sensitivity analysis include:

• Number of IT support staff
• Skill level of IT support staff
• Adoption level of automation tools (e.g., ticketing systems, AI-driven diagnostics)
• Average complexity of incidents

Determine Ranges and Distributions

For each key input, the team defines plausible ranges based on historical data and expert judgment. For instance, the number of staff could vary from current levels up to a 20% increase, considering budget constraints.

Choose a Method for Analysis

The team opts for a combination of One-at-a-Time (OAT) and Monte Carlo simulation methods. This approach allows them to isolate the impact of individual variables and assess the overall effect of varying all inputs within their distributions.

Execute the Analysis

Using simulation software, the team systematically varies each input within its defined range while holding others constant and then conducts a Monte Carlo simulation to understand the probabilistic impact of all inputs changing simultaneously.

Analyse Results

The analysis reveals that while increasing the number of IT support staff and enhancing their skill levels both significantly reduce resolution times, the adoption of automation tools has the most substantial impact, especially for high-complexity incidents.

Interpret and Communicate Findings

The findings suggest prioritising investments in automation tools and targeted training to upskill the IT staff. These insights are presented to stakeholders, highlighting the potential reduction in incident resolution time and improvements in service quality.

Make Informed Decisions

Guided by the sensitivity analysis, the IT service provider decides to allocate the budget towards implementing advanced automation tools and developing a comprehensive staff training program focusing on handling complex incidents efficiently.

Through this example, sensitivity analysis enables the ITSM team to strategically direct their improvement efforts, ensuring they invest in areas with the highest potential impact on enhancing service delivery and customer satisfaction.

Tornado Diagram

A tornado diagram is a visual representation used to display the results of a sensitivity analysis, highlighting how variations in input variables affect a specific outcome of interest. It's called a "tornado" diagram because the final chart resembles the shape of a tornado—wide at the top and narrow down at the bottom. This diagram is particularly useful in decision-making processes, risk management, and financial analysis, providing a clear view of which factors have the most significant impact on the outcome. Here is an example:

Project Budget Analysis

Let's consider a project management scenario where a project manager is conducting a sensitivity analysis to understand which factors most significantly affect the project's total budget. The key inputs (variables) might include:

• Cost of materials
• Labour rates
• Project duration
• Equipment costs
• Subcontractor costs

For each of these inputs, the project manager estimates a range of possible values and assesses the impact of each variable on the total project budget. The sensitivity analysis reveals how much the total budget could change with variations in these inputs.

Creating the Tornado Diagram

Identify Key Variables: The project manager lists the variables analysed, such as labour rates, cost of materials, etc.

Range of Impact: For each variable, the difference between the highest and lowest potential impact on the project budget is calculated.

Rank Variables: The variables are ranked from the one with the highest impact on the budget at the top, to the one with the least impact at the bottom.

Draw the Diagram: A horizontal bar is drawn for each variable. The length of the bar represents the range of impact (from negative to positive) that the variable has on the project budget. The variable with the widest range (greatest impact) is at the top, creating a wide start to the "tornado," and the one with the narrowest range (least impact) is at the bottom, creating the narrow end.

Interpretation

The tornado diagram visually communicates that changes in the cost of materials and labour rates have the most significant potential impact on the project budget, suggesting these areas should be a focus for cost management and risk mitigation strategies. Meanwhile, changes in subcontractor costs have the least impact, indicating that this area might not need as tight a focus in the risk management plan.

This visualisation helps the project manager and stakeholders quickly identify where to focus their efforts and resources to control costs and manage risks effectively. It simplifies complex data into an accessible format, aiding in strategic planning and decision-making.

Sensitivity Analysis vs Scenario Analysis

Sensitivity analysis and scenario analysis are both valuable tools used in decision-making processes, risk management, and planning across various industries. Despite their similarities in purpose, they differ significantly in their approach and application. Understanding these differences is crucial for selecting the appropriate method based on the specific needs of a project or analysis.

Sensitivity Analysis

Definition: Sensitivity analysis investigates how changes in the inputs of a model influence the outcome. It's primarily used to determine the "sensitivity" of the output to variations in each input variable.

Approach: This analysis typically varies one input at a time (One-at-A-Time, OAT method) to observe the effect on the output while keeping other inputs constant. Advanced methods can also vary multiple inputs simultaneously to assess their combined impact.

Purpose: The main goal is to identify which variables have the most significant impact on the outcome, helping to prioritise focus areas in decision-making and risk management.

Application: Widely used in financial modelling, engineering, and environmental studies to assess the robustness of models and to identify key drivers of risk and performance.

Scenario Analysis

Definition: Scenario analysis examines the effects of different predefined scenarios on the outcome. Each scenario represents a set of changes in multiple variables to simulate various possible futures.

Approach: Instead of varying one variable at a time, scenario analysis changes several inputs together based on specific assumptions to create a scenario. It evaluates the outcomes under different scenarios, such as a "best case," "worst case," and "most likely case" scenarios.

Purpose: It helps in understanding the broader implications of complex, interrelated changes and in preparing for various potential futures by evaluating the impact of different scenarios on the outcome.

Application: Frequently used in strategic planning, risk management, and policy analysis, allowing organisations to plan for different possible futures and develop strategies that are robust across various scenarios.

Key Differences

Focus: Sensitivity analysis focuses on the impact of individual variables changing one at a time or in combination, while scenario analysis looks at the outcome of comprehensive, predefined scenarios involving simultaneous changes in multiple variables.

Complexity of Interactions: Scenario analysis can capture complex interactions between variables in a way that sensitivity analysis might not, as it considers the combined effect of changes in multiple variables.

Outcome Interpretation: Sensitivity analysis is particularly useful for identifying critical inputs and for prioritising risk management efforts. Scenario analysis, on the other hand, is better suited for strategic planning and preparing for a wide range of potential future events.

Choosing between sensitivity analysis and scenario analysis—or using them in conjunction—depends on the specific goals of the analysis, the complexity of the system or model, and the type of decisions or plans that need to be informed by the analysis.

The Benefits of Sensitivity Analysis

Sensitivity analysis offers several benefits across various fields such as finance, engineering, environmental science, and policy making. Its value lies in providing insights that help in decision-making, model validation, and risk management. Here are some key benefits of performing sensitivity analysis:

Identifying Key Drivers

It helps identify which variables have the most significant impact on the outcome of a model or system. Understanding these key drivers allows organisations and researchers to focus their efforts and resources on the factors that truly matter, enhancing efficiency and effectiveness.

Enhancing Understanding of the Model

By systematically varying inputs and observing the outputs, sensitivity analysis deepens the understanding of the interrelationships within the model. It can reveal how inputs interact with each other and with the model structure, leading to a better grasp of complex systems.

Informing Decision Making

The insights gained from sensitivity analysis can be invaluable in making informed decisions. Knowing how different factors affect outcomes enables decision-makers to weigh options more accurately, consider alternative strategies, and mitigate risks effectively.

Supporting Risk Management

Identifying how sensitive an outcome is to changes in inputs can highlight potential risks. This knowledge is crucial for developing strategies to manage those risks, whether by taking proactive steps to prevent undesirable outcomes or by preparing contingency plans.

Enhancing Communication and Transparency

The results from sensitivity analysis can be used to communicate the robustness and reliability of a model to stakeholders, providing a transparent basis for decisions. This can be especially important in policy-making, environmental management, and public health, where stakeholder trust and buy-in are critical.

Facilitating Model Validation and Calibration

Sensitivity analysis can play a key role in model validation and calibration by identifying inputs that significantly influence the output. This can guide the collection of more accurate or detailed data for those inputs, improving model accuracy and reliability.

Preparing for Uncertainty

In an ever-changing world, the ability to understand and prepare for uncertainty is invaluable. Sensitivity analysis allows organisations and policymakers to explore how changes in the external environment could impact their operations or objectives, helping them to be better prepared for future changes.

Supporting Cost-Effective Analysis

By highlighting which variables are most important, sensitivity analysis can make further data collection and analysis more cost-effective. Instead of spreading resources thin over many variables, efforts can be concentrated on the few that really matter, optimising the use of limited resources.

The Limitations of Sensitivity Analysis

While sensitivity analysis is a valuable tool for understanding the dynamics within models and systems, it also has its limitations. Recognising these limitations is crucial for interpreting the results accurately and using them effectively in decision-making processes. Here are some notable limitations of sensitivity analysis:

Dependence on Accurate Models

The effectiveness of sensitivity analysis is heavily reliant on the accuracy and completeness of the underlying model. If the model does not accurately represent the real-world system or process, the insights gained from sensitivity analysis may be misleading or incorrect.

Complexity in Handling Interdependent Variables

Sensitivity analysis can become complex and less intuitive when dealing with models that have a high degree of interaction between variables. The one-at-a-time (OAT) approach might not capture the nonlinear interactions or the combined effects of multiple variables changing simultaneously.

Difficulty in Selecting Appropriate Ranges and Distributions

Determining the right ranges for input variables and their probability distributions can be challenging, especially in the absence of sufficient data or clear guidance. Incorrect assumptions about ranges and distributions can lead to inaccurate conclusions.

Time and Resource Constraints

Performing comprehensive sensitivity analysis, an especially global sensitivity analysis that explores the full parameter space can be computationally intensive and time-consuming. This may limit its application in projects with tight deadlines or limited computational resources.

Risk of Oversimplification

There's a tendency to oversimplify complex systems to make sensitivity analysis more manageable, which can lead to overlooking important aspects of the system. This oversimplification might result in incomplete or skewed insights.

Interpretation Challenges

The results of sensitivity analysis can sometimes be counterintuitive or difficult to interpret, especially for complex models with many interacting variables. Misinterpretation of the results can lead to incorrect decisions or strategies.

Limited Predictive Power

Sensitivity analysis can indicate how changes in inputs affect outputs but does not necessarily predict future states of the system. It is more about understanding the model's behaviour under different conditions than about forecasting specific outcomes.

Potential for Paralysis by Analysis

Focusing too much on the results of sensitivity analysis may lead to "paralysis by analysis," where decision-making is delayed due to overemphasis on analysing all possible scenarios and variables. This can hinder timely action and responsiveness.

Despite these limitations, sensitivity analysis remains an indispensable tool in many fields, offering significant insights into system behaviours and decision impacts. The key is to use it judiciously, in conjunction with other analytical methods, and to be mindful of its limitations when interpreting and applying its results.

Final Notes on Sensitivity Analysis

In conclusion, sensitivity analysis is a powerful tool that, when applied correctly, offers invaluable insights into the dynamics of models and systems, aiding in more robust and informed decision-making. However, it is essential to be aware of its limitations and the context in which it is applied to ensure the results are interpreted accurately and used effectively.

Remember, always validate your model with real-world data where possible before conducting sensitivity analysis. This step ensures that the model accurately reflects the system being studied, thereby enhancing the reliability and relevance of the sensitivity analysis results. Integrating this validation phase into your approach can significantly mitigate some of the limitations associated with model accuracy and representativeness.