How Kaiser Permanente is Using Causal AI to Revolutionize Patient Care
Founded in 1945, this non-profit organization was born from an innovative healthcare program for shipyard workers during World War II. Today, it’s one of the largest managed care organizations in the United States, serving over 12 million members. What sets Kaiser Permanente apart is its revolutionary model that combines health insurance, hospitals, and medical groups under one umbrella.
This integrated approach not only streamlines patient care but also provides a rich environment for data-driven insights and improvements. At the recent Causal AI Conference, Kaiser Permanente’s data science team shared how they’re leveraging this integrated model and causal inference to drive meaningful change in healthcare delivery and patient outcomes.
At the recent Causal AI Conference, Kaiser Permanente’s data science team shared how they’re leveraging this integrated model and causal inference to drive meaningful change in healthcare delivery and patient outcomes.
What follows is an edited transcript of the talk, organized into sections for clarity.
Speakers: Naveed Sharif, Director of Data Science & Web Analytics, and Jeff Groesbeck, Staff Data Scientist, Kaiser Permanente
This transcript has been lightly edited for length and clarity.
Introduction and Kaiser Permanente’s Market Presence
My name is Navid. I lead a data science team within Kaiser Permanente. I’m excited to share with you how we’re leveraging causal inference to design and improve some of the digital products that we are currently working on. We’re in the digital department, and specifically, we’re going to see the application of causal inference within a fully integrated healthcare system.
The interesting statistic here is that in Northern California, one out of three individuals are currently insured through Kaiser’s healthcare plan. So it’s a huge dominance within this Northern California region.
From this presentation, the three things that we’ll probably learn are:
- How we’re using causal inference to design digital products
- How our focus in causal inference is patient care and the quality of care that we’re providing
- How it’s very important for us to safeguard our interventions. A false positive within a healthcare system has a huge impact on our patients.
The Dynamics of Healthcare in the United States
Before diving into causal inference and how we’re applying it within Kaiser Permanente, I think it’s really important to first level set by understanding the dynamics of healthcare within the states. I want to explain it through three core ideas:
1. Healthcare evolves dramatically as we age. For example, in my own personal experience, I have a young son at home. A lot of the experience has been around going into a pediatric office, looking at growth charts, and getting immunization shots. The second thing is I have aging parents that aged into Medicare. Medicare is very different coverage from something that maybe your employer is giving to you. And then the third thing is there are some unexpected challenges that do happen. My father was diagnosed with Parkinson’s, and so searching for specialist care is a challenge. It took us a lot of days to find out what is the right type of specialty care for my father.
2. There are some really important decisions that you have to make that impact the quality of care that you’re getting and the type of care that you’re receiving. The first thing is the insurance selection. That insurance selection dictates the type of providers you have access to, the type of services you actually have, and then also impacts the type of finances that might be due based on the services that are rendered to you. The other piece is the provider selection. You can go through a fully integrated healthcare system like Kaiser Permanente, you can go through an in-house network like Sutter Health or John Muir, or you can also go through a private practice.
3. Understanding really how fragmented the healthcare system is. We talk about all these areas from shopping to accessing care to delivery to managing care. A lot of organizations are not under one umbrella. They don’t talk to each other. They all have their own kind of initiatives and objectives as well. But us as patients, we don’t really differentiate that as corporations tend to do.
Kaiser Permanente’s Fully Integrated Healthcare System
So what is a fully integrated healthcare system? It’s everything that we just saw like insurance, shopping for insurance, the hospitals and facilities where you’re receiving care, the clinicians and the doctors all under one umbrella.
Now there are some advantages and some challenges. The first advantage is enhanced coordination. Imagine you’re a patient, you have some issues with your eyes and you go and see an optometrist. That optometrist can actually log into the electronic medical records, your charts, and they can actually see the notes that an ophthalmologist may have entered for that patient. That ophthalmologist might be able to also access the logs or notes that maybe their primary care physician had entered. So all that’s collectively within your charts, and there’s that enhanced coordination that exists under an umbrella of care such as Kaiser.
The other advantage is the data richness. This truly does benefit our clinicians. We talked about how they’re able to access some of that information. It also benefits our researchers. But I want to double down on this with how rich our data is within a fully integrated healthcare system. We talked about those different components from premiums and claims – all of that is managed within a fully integrated healthcare system. We do have access to that information. We talked a little bit about your charts and the electronic medical records – all of that also you’re able to access. There’s also digital data, the interaction that happens digitally on our webpage. You’re able to access that, and then also just the hospitals, the facilities that you have access to, what type of clinicians you saw – those data points are being captured as well.
Now the other key thing here is, and this is in most healthcare systems, there’s not that much churn. Usually, you stick with your doctor, you’re happy with your doctor, you’re not really moving around. And so we have these large longitudinal datasets of individuals literally from their cradle to their grave. I think that’s a very unique thing within a fully integrated healthcare system as well.
Now the challenges do exist. The first one, and this is probably what we’re most known for, specifically Kaiser, is the geographic constraints. If you are in a location and there are not that many hospitals in that location that you can have access to, it might be very difficult to get some sort of specialized care that you would want to get for whatever condition that you’re going through.
The second thing is the complex management. Since these three pillars exist under one umbrella, there is a lot of conflicting interest within these three areas of that umbrella. So it is a little slow to push innovation or technology within that area, and that does exist and it is a hurdle for sure from a bureaucratic perspective.
Leveraging Causal Inference in Digital Healthcare
Let’s start jumping into some things that we’re doing with causal inference within the digital department. The first thing is that, just like a lot of folks talking about their organizations, we do have an AB testing environment. A lot of the digital products that we do launch are anchored to causal insights. It’s based on a Bayesian framework. One of the big reasons why we chose the Bayesian framework was because of a lot of the contextual information that doctors or clinicians might have that you could pass onto priors.
The other thing to understand about these AB tests that we do run is that we’re in digital, so we get to scale across those three arms, the three verticals of healthcare. When you’re able to scale across those three verticals, you’re able to really expose a lot of different metrics and impact that you have. So from an operational perspective like call centers, from a clinician perspective like burnout, from a patient care perspective, from an engagement perspective. And because you have that much visibility, you actually can also place guardrails. Guardrails are really, really important for us within Kaiser Permanente.
Now as you guys know, you can’t run an experiment on everything. In fact, there’s a lot of limitations in healthcare, and so quasi-experiments is really where we start to work into. Quasi-experiments are actually pretty effective within a fully integrated healthcare system within Kaiser because we do have not that much censored data. Data does exist, it is censored, but we get to follow these patients again throughout their lives.
For example, for my father or my parents who’ve aged into Medicare, when you’re transitioning out from your previous insurance plan into Medicare, let’s say if you’re under Kaiser, you can follow that. And so you might have not needed to change doctors or clinician staff or system. That is all being followed. Let’s say my parents then wanted to do some physical therapy, that engagement with that physical therapist is also captured. So all those events are actually captured within an integrated model.
Now confounders do exist, or unobservables do exist, but there is, I think, a lot of ways to control for some of the confounders. Again, we do have a lot of access to those data points.
Addressing Clinician Burnout
Before I hand the mic to Jeff, who’ll get into a specific use case where we’re going to really start to surface some of the advantages and challenges within Kaiser Permanente or a fully integrated healthcare system, I want to talk about what we’ve been spending a lot of time on.
Post-COVID, there has been a huge influx of utilization within our healthcare system, and so clinician burnout is a real thing. One of the things that we want to measure is how do we actually reduce that burnout and then also what type of impact might it have on our patients. Some of the things that we do are like how can we reroute messages that are maybe sent to your primary care physician.
Something else to keep mindful here is that primary care physicians are kind of like the gatekeepers, and so they are getting a lot of messages or they’re getting contacted quite a bit. So if we can actually reroute some of the messages like to an optometrist or ophthalmologist or a dermatologist, that should reduce the amount of work that a PCP might have.
Case Study: Prescription Notifications
Jeff Groesbeck: Awesome, thank you Navid. All right, let’s dive in. So we’re going to talk about prescription notifications. A long-standing question from the pharmacy arm of Kaiser was about the impact of refill reminders, specifically around medication adherence but also medication over-utilization.
Just to give you a brief context, refill reminders are either SMS, push, or email notifications sent to members when their prescriptions are eligible to be refilled. This was introduced as an opt-in feature on kp.org to members in late 2020.
In terms of the data that we’re going to use for this study, we’re going to focus on a single maintenance medication used to treat hypertension. It’s one of the most prescribed medications at Kaiser Permanente, and it’s dispensed with a 100 days supply typically.
In terms of scope, we’re looking in Southern California. So like Navid mentioned, we have many different regions. We’re going to use just Southern California for 2019 and 2022, and we’re also going to use digital notifications sent to these members for that prescription again for the same time frame.
In terms of the analysis, we’re going to use a classic differences-in-differences approach. Just to make everything very concrete, in the treatment we have members who have opted into refill notifications within 100 days of that rollout on kp.org. And then for the control, we’re going to use the never-treated members who never sign up nor receive notifications through any channel.
In terms of that time period, the feature was released December 20 of 2020, and so we’re going to be splitting our time frame, our years, into 100-day periods pre and post. Obviously, one of the important parts of this feature, which we’ll talk about, is that this is an opt-in feature, and so members are opting in. They’re choosing to actually enroll in these refill reminders.
In terms of our outcomes, I mentioned adherence and over-utilization. So adherence, very simply, is a member who has more than zero days supply when they refill their prescription, or if they just refill that prescription on that 100th day, we could consider them adherent.
For over-utilization, you can obviously think about this in many different ways. After conversations with Pharmacy, we decided to land on 40-day supply. So if a member has more than 40 days supply and then they refill that prescription, we consider that medication over-utilization.
Just to be clear, over-utilization doesn’t necessarily mean a member is taking too much of that medication. Especially in this case, we’re talking about a maintenance medication, it’s very standard. It’s more likely that they just have more medication on hand than necessary.
In terms of the business rule, how are these refill reminders actually sent? How do members actually refill these prescriptions? Business rules allow refills at 80% utilization from their most recent fill, which is obviously going to be very essential. So if a prescription is written for 100 days supply like the ones we have here, a member can choose to refill that prescription after 80 days. And if a member signs up for those notifications, they will receive a reminder at that 80th day that their prescription is available to refill.
Let’s jump straight into the average treatment effects. So on the top, we have over-utilization, and in the bottom plot, we have medication adherence. You can see for both of these metrics, the pre-treatment trends are all within zero, and then as soon as the rollout happens, both of these metrics kick up. Adherence is pretty immediate as you expect given how that’s constructed, whereas over-utilization takes a little bit more time.
But in terms of the magnitudes we’re talking about, we’re talking about somewhere between 10 and 15 percentage points of members who are either increasing their medication adherence or over-utilizing that medication. So it’s relatively similar actually in terms of magnitudes.
Long story short is that refill reminders are causing both adherence and over-utilization. Bringing this back to the pharmacy, what does the pharmacy group want to accomplish? They want all the adherence without any of the over-utilization.
With this evidence, essentially what they’ve been able to do is very concretely argue that the first and most obvious thing is we need to change that business rule that I mentioned here, which is allowing refills at 80% utilization. So right now we’re going through tests in terms of how we actually set up this rule to try and achieve that hopefully achievable goal of getting adherence without that over-utilization.
The other thing that the business has been able to do with this is that now that we’ve validated we know that refill reminders are helping adherence, that’s just step one. We’ve only just now learned that this feature has helped improve adherence, and so this has basically led the business to adopt different features or try and test different features to try and squeeze this a little bit more and try and see if we can increase that adherence.
In terms of examples, we’re talking about personalizing the timing of these refill reminders to try and get the member to order that prescription at like the best time of day for them.
So yeah, we’re excited about this work and continuing to apply these methods in this space. You can see this is just, you know, we’re very much scraping the surface here in terms of the capabilities of this data and the integrated model using causal inference.
The Future of Causal AI in Healthcare
Kaiser Permanente’s approach to causal inference within their integrated healthcare model showcases the potential for data-driven improvements in patient care.
As we look to the future, it’s clear that causal AI offers significant advantages over traditional AI and statistical approaches in healthcare.
Evidence-based interventions: By moving beyond mere correlations, causal AI allows healthcare providers to design more effective interventions based on a deeper understanding of cause-effect relationships.
Personalized medicine: As seen in the prescription notification case study, causal AI can tailor treatments and communications to individual patients. Unlike one-size-fits-all approaches, causal AI can account for individual patient characteristics and behaviors, enabling personalized care plans that boost patient engagement and health outcomes.
Operational optimization: Causal AI excels at identifying the most impactful factors in complex systems. In healthcare, this means it can pinpoint the root causes of issues like clinician burnout or inefficient resource allocation more accurately than traditional methods, leading to more effective solutions and streamlined operations.
Ethical decision-making: By providing clearer insights into the impacts of various interventions, causal AI can support more ethical and effective policy decisions in healthcare.
Causal AI has the potential to make healthcare more efficient, more personalized, and ultimately more effective in improving patient outcomes and overall public health.