Causal AI: from predicting to shaping the future

Introduction

Businesses ultimately want to do more than just predict the future — they also want to actively shape it. To do so they need intelligence tools to make the right decisions, plans and strategies.

Perhaps surprisingly, this is beyond what’s achievable with today’s predictive analytics or state-of-the-art machine learning (ML) systems. While these systems can make predictions, they are inherently limited when it comes to designing a set of actions or strategies to improve business outcomes. Conventional ML algorithms cannot anticipate the consequences of so-called “interventions”: changes to a causal system that are made from outside that system.

Businesses need a new technology to evaluate potential interventions, optimise decisions and take control of their future. That technology is Causal AI.

What If?

What happens to revenue if we halve the price? What effect will increasing financial inclusion have on poverty alleviation? Will drinking less coffee improve my sleep? These are all questions about what happens to a system when some variable in that system is deliberately adjusted to take on a certain value. They are examples of “interventional” questions, which are ubiquitous in business, public policy and daily life.

Such questions can in principle be answered experimentally. But while experimenting is critically important, it can only go so far. Running randomized controlled trials, the gold standard of experimentation, can be slow, expensive or fraught with ethical challenges. Even in the digital world, where technology companies can run huge numbers of A/B tests with relative ease, experimentation has its limits.

When businesses want to reduce the costs, complexity and lag times that come with running experiments, or when experimenting simply isn’t a viable option, they will need to turn to statistical and computational methods.

Interventions Go Beyond Curve-Fitting

However, conventional ML systems fall short when it comes to answering interventional queries. Consider a telecommunications provider that’s identifying which customers its call centre should target with an intervention, outreach by a salesperson, in order to prevent “churn” (abandonment of the service).

It’s common practice to use conventional ML to simply predict and then target the customers at the highest risk of churn. But targeting likely churners is pointless if the interventions have no impact and the customers plan to quit the service regardless.

By treating interventional problems as prediction problems to be solved with conventional ML, businesses are needlessly losing profitability

A Harvard Business School study demonstrates that it’s more effective to allocate interventions to customers who are most responsive to them. In fact, the study found that the overlap between the highest-risk churners and the group that are most responsive to intervention was just 50%. By treating an interventional problem as if it were merely a prediction problem to be solved with standard ML, businesses are wasting resources and needlessly losing customers.

ML algorithms can also be deployed in churn management programs to predict which customer segments tend to renew their subscription when called by salespeople. That is, they can identify correlations between customer segments and renewal rates, conditional on an intervention. But standard algorithms can’t discern whether these customers would’ve renewed or upgraded anyway. Accordingly, the telco can’t prioritise the highest return-on-investment sales calls.

These results come as no surprise to AI researchers who’ve shown that interventional questions can’t be answered by simple predictive models that merely analyze patterns of associations between variables. Since ML algorithms are limited to pattern-matching it follows that they’re blind to the effects of interventions.

Interventions Require Causal Knowledge

Causal AI provides the only credible way to evaluate interventions in silicon. It starts by penetrating beneath the observable data to the hidden nuts-and-bolts processes that are generating that data. Given access to big data, the AI can leverage increasingly sophisticated causal discovery techniques to learn about the system. But, unlike conventional ML, Causal AI can also function in small data environments, with human experts giving the AI the benefits of their market insight.

Equipped with a causal model, the AI can run a kind of virtual experiment. Returning to our churn prevention example, the AI creates groups of counterparts and customers with similar profiles. It then adjusts for the factors that led to the telco contacting some of these customers and not others. This enables the AI to isolate the effect of the intervention on propensity to churn without contamination from the factors that led to the customer receiving the sales outreach. More broadly, Causal AI can draw on a large, rapidly expanding set of causal inference algorithms that are appropriate for evaluating different types of interventions.

Causal AI can run virtual experiments at the click of a button

With a sufficiently rich causal model, the AI can infer the effects of an intervention purely by analysis, without the need for costly or infeasible experiments. Users get to conduct experiments at the click of a button — instantaneously and without cost. Even when this isn’t possible, Causal AI radically reduces uncertainty about the potential effects of an intervention.

Causal AI for Business Interventions

Causal AI helps businesses to design optimal interventions and make better decisions. By assessing interventions ahead of time, businesses can make mistakes in a risk-free virtual environment. This eliminates expensive trial-and-error learning, making it more likely that good decisions are taken at the first time of asking. Interventions are also adaptive to dynamically shifting markets and priorities. And otherwise resource-intensive decision-making processes are more streamlined.

It would be a mistake to think of Causal AI as a surrogate for experimentation — instead, it augments it. Routine experimentation remains essential for gathering market intelligence. Causal AI then enhances this intelligence by enabling knowledge workers to run high volumes of virtual experiments, freely and instantaneously.

Humans can partner with the AI to request analyses to aid intervention design. For some processes, the AI can autonomously make business improvements that optimize for key performance indicators.

By assessing interventions ahead of time, optimal decisions can be taken without costly trial-and-error learning

Beyond churn management, the value of interventional analysis applies to a broad sweep of resource allocation problems, pricing decisions and investment actions. An airline shaping demand through dynamic pricing. A retailer optimizing promotions by factoring in the effects of price discounting on the consumer shopping basket. An asset manager staggering its trading actions in order to mitigate execution costs incurred by the market influence of its trades. These are all interventional problems demanding Causal AI.

The power to evaluate interventions extends far beyond these handful of examples. Beyond asking “what happens if we do…?”, businesses can also ask, “what happens if our competitor does…?”, or “what happens if the central bank does…?” By making these third-personal interventional queries, users can surface insights to support modelling competitive dynamics and scenario planning.

The point of business intelligence is to ultimately make good decisions and take action. However, most conventional ML systems can only provide limited decision support, because they can’t anticipate how actions change the environment. Accordingly, business decisions are currently based on a patchwork of flawed predictive analytics, human intuition and costly trial and error. Causal AI has the power to change this, by transforming intervention design and decision-making from an art into a science.

Stay tuned for an article in which we introduce another signature feature of Causal AI: artificial imagination. Causal AI can reimagine the past, explaining why events unfolded as they did and exploring alternative branches that might have been. This helps unearth hidden layers of insight that can help organizations to better navigate the future.