PyData Global 2024

Objective

This talk will provide a hands-on exploration of Judea Pearl's causal ladder. This will be made tangible using a simple demand prediction problem, and we will explore how different business questions map on to different levels of the causal ladder: association, intervention, and counterfactuals. We will look at short code examples of models for each of the levels, enabling attendees to understand how to implement the causal concepts in code.

Central Thesis

Traditional forecasting models rely heavily on correlation, which limits their ability to inform strategic decisions. By leveraging causal inference, data practitioners can move beyond correlation to predict the impact of interventions (like changes in pricing or advertising spend) and evaluate counterfactuals (such as evaluating the effectiveness of a promotional campaign). Using PyMC and other packages, I will illustrate how each step of the causal ladder can help answer increasingly complex business questions.

Outline and Time Breakdown

Introduction (5 minutes)

Overview of the challenges in traditional demand forecasting.
Introduction to Judea Pearl’s causal ladder and its three rungs:

• Association: Observational modeling.
• Intervention: Action via the do-operator.
• Counterfactuals: What-if analysis.

Level 1: Association (5 minutes)

Demonstrating basic forecasting models in PyMC to predict future demand based on historical data.
Discussing how these models use correlation to make predictions under normal business conditions.
Example: Forecasting sales based on past data like price, advertising, and seasonal factors.

Level 2: Intervention (8 minutes)

Introducing the concept of interventions and the do-operator in PyMC.
Example: Simulating an intervention where the advertising spend is doubled, breaking the correlation with other factors, and predicting the outcome using causal inference.
Showcasing how interventions can inform better business decisions (e.g., budgeting for ad campaigns).

Level 3: Counterfactuals (8 minutes)

Moving to counterfactuals: Predicting what would have happened under a different set of historical actions.
Example: Predicting the impact of a past price promotion by retrodicting sales in the counterfactual absence of a price promotion.
Using PyMC’s do-operator to compute these counterfactual scenarios, helping businesses evaluate past strategies and optimize future decisions.

Q&A (4 minutes)

Addressing audience questions.

Key Takeaways:

• Attendees will learn how to apply the three levels of Pearl's causal ladder to a common business problem: demand forecasting.
• Practical demonstration of how to use PyMC’s causal tools (including the do-operator) for forecasting, strategic decision-making, and evaluating what-if scenarios.
• Attendees will leave with an understanding of how causal inference can add value beyond correlation-based methods in real-world business settings.

Audience

This talk is aimed at data scientists, machine learning engineers, and business analysts. A basic understanding of Python and Bayesian statistics will be helpful but not strictly required, as the talk will focus on the application of tools rather than mathematical derivations.

Why this is interesting

The ability to forecast is critical to business success, but many forecasts fail to account for causal relationships, leading to suboptimal decisions. By moving beyond correlation and incorporating causal reasoning, attendees will learn to make more informed and actionable business decisions. The use of open-source tools like PyMC makes this accessible and applicable to a wide range of industries.