Shingles Shot and Dementia: Could one vaccine protect your brain?

[Kristin] (0:00 - 0:05)
This family of virus, oh, it also includes Epstein-Barr virus, Regina.
 
[Regina] (0:05 - 0:19)
Oh, which is actually another connection to sex, right? Because that, if I remember correctly, is the virus that gives you mononucleosis, sometimes known as the kissing disease, because it spreads through saliva, if I remember.
 
[Kristin] (0:20 - 0:20)
That's right.
 
[Regina] (0:21 - 0:24)
We are hitting all of the glamorous parts of sex today.
 
[Kristin] (0:30 - 0:53)
Welcome to Normal Curves. This is a podcast for anyone who wants to learn about scientific studies and the statistics behind them. It's like a journal club, except we pick topics that are fun, relevant, and sometimes a little spicy.
 
We evaluate the evidence, and we also give you the tools that you need to evaluate scientific studies on your own. I'm Kristin Sainani. I'm a professor at Stanford University.
 
[Regina] (0:53 - 0:59)
And I'm Regina Nuzzo. I'm a professor at Gallaudet University and part-time lecturer at Stanford.
 
[Kristin] (0:59 - 1:04)
We are not medical doctors. We are PhDs. So nothing in this podcast should be construed as medical advice.
 
[Regina] (1:05 - 1:09)
Also, this podcast is separate from our day jobs at Stanford and Gallaudet University.
 
[Kristin] (1:10 - 1:17)
Regina, today we're going to talk about shingles and the shingles vaccine. And it's not the sexiest topic in the world.
 
[Regina] (1:18 - 1:41)
It is not. In fact, I mentioned to a friend that we were doing shingles vaccine for this episode. And he said, how are you going to work sex into this one?
 
And I'm thinking, okay, it's this painful, ugly, red rash. And who wants to have sex with you when you have shingles? No one.
 
Therefore, get the vaccine and have more sex.
 
[Kristin] (1:42 - 2:04)
Okay, good. We have now gotten sex into the episode. Check.
 
Well, we're talking about the shingles vaccine because not only does it prevent shingles, but there have been some recent papers out that provide evidence for the claim we're looking at today. Which is that the shingles vaccine reduces the risk of dementia later in life.
 
[Regina] (2:05 - 2:12)
Which are two things that I never thought would be related. Shingles and your brain.
 
But we will talk about that mechanism later.
 
[Kristin] (2:13 - 2:43)
Absolutely. Today, we're going to unpack two 2025 papers about the shingles vaccine and dementia, both of which got a lot of media attention. And Regina, as we've talked about before, the gold standard study design for proving causation is a randomized trial.
 
But we don't have randomized trials here because it would be unethical to withhold shingles vaccine from people, since we know that the shingles vaccine is beneficial and prevents you from getting shingles.
 
[Regina] (2:43 - 3:28)
Right. So to get around this little problem, these papers use something called causal inference methods, which is basically like a toolbox of stats techniques that we turn to when we can't actually do a randomized trial like here. And we'll talk about some of the tools in the toolbox today, including, Kristin, natural experiments, propensity scores, and inverse probability of treatment weighting.
 
Which is like super fun, jargony names. As a side note, we talked about another causal inference method way back in the alcohol episode. That one was called Mendelian randomization.
 
So, Kristin, we're getting to revisit causal inference again.
 
[Kristin] (3:29 - 3:51)
You know, Regina, I kind of hate that name, causal inference methods. It makes it sound like we're doing some kind of statistical voodoo magic to magically make it possible to get causality out of observational data. But really, these methods come with a lot of caveats and assumptions, and they're not as good as actual randomization.
 
[Regina] (3:51 - 3:56)
Right.
 
Statistics, not magic, as we have said many times before.
 
[Kristin] (3:56 - 4:07)
That's right. So, Regina, let's start by talking about what shingles is. It's caused by the varicella zoster virus, and that's the same virus that causes chickenpox, actually.
 
[Regina] (4:07 - 4:14)
It sounds like a great name for a punk rock band. Varicella zoster and the chickenpox.
 
[Kristin] (4:16 - 4:38)
Well, this is the interesting thing, Regina. If you've ever had chickenpox, the virus never really leaves your body. It hides out dormant in your nerve cells for decades, and later in life it can wake up again, travel along those nerves to the skin, and cause this painful, blistering rash, which is what shingles is.
 
Oh, this is a weird virus.
 
[Regina] (4:38 - 4:49)
It's just the same virus that you had when you were a kid, only now it's coming back to haunt you in your middle age.
 
[Kristin]
Yeah.
 
[Regina]
So, I've never had shingles before.
 
Do you know what it actually feels like? Have you had it?
 
[Kristin] (4:49 - 5:23)
I've never had it, but I did look it up, and it's described as a burning, tingling pain. So, your skin actually hurts before the rash appears, and then a day or two later, you get a red stripe or patch on one side of your body, and then the blisters pop up, and apparently they're really painful, and it hurts even to wear clothing.
 
[Regina]
Well, there you go.
 
That sounds good. That's sexy.
 
[Kristin]
It usually lasts a couple of weeks, but for some people, the nerve pain can linger for months.
 
So, yeah, not fun and not sexy.
 
[Regina] (5:23 - 5:45)
Okay, so shingles is not an aphrodisiac. You've convinced me. So, almost everyone born before the mid-90s had chickenpox, right?
 
So, they carry this varicella zoster virus. They are at risk for shingles. Kristin, that includes us.
 
We were born before the mid-1990s, I think, considering we met in grad school in 96.
 
[Kristin] (5:45 - 5:52)
Yeah, we will admit to being born before the mid-1990s, Regina. So, did you have chickenpox as a kid?
 
[Regina] (5:52 - 5:58)
I did. It was ninth grade, super itchy. That is all I remember.
 
It was gross. Did you have it?
 
[Kristin] (5:58 - 6:26)
Yes, and I also didn't get it until high school, which was kind of late, actually.
 
[Regina]
Clearly, we didn't socialize enough with other kids.
 
[Kristin]
Or we both had overly protective parents who kept us away from germs.
 
Okay. I do distinctly remember getting the chickenpox because it was my sophomore year in high school, and I got it right before the qualifying meet for the New Hampshire State Championships in outdoor track. So, I didn't get to run the state meet that year.
 
[Regina] (6:26 - 6:27)
Oh, bummer.
 
[Kristin] (6:27 - 6:43)
It was a bummer, but funny story, they actually changed the state rules because of me. So, if you were out for the qualifying meet and had one of the top times in New Hampshire, they would still let you compete at state. So, it didn't help me, but it did help other people after me.
 
[Regina] (6:43 - 6:47)
Aw, you helped change history, but did you ever get to compete in state then, or that was it?
 
[Kristin] (6:48 - 6:54)
Yes, I did go on later to win some state championships in track and cross country in the tiny state of New Hampshire, yes.
 
[Regina] (6:54 - 7:06)
Oh, Kristin, you are a rock star.
 
[Kristin]
Thanks, Regina.
 
[Regina]
Okay, so chickenpox.
 
Anyone who had chickenpox as a kid is at risk for shingles later in life, and that is a lot of people we're talking about.
 
[Kristin] (7:07 - 7:17)
It is. I didn't realize this until researching this episode, but about one in three Americans will get shingles in their lifetimes, which means it's super common.
 
[Regina] (7:18 - 7:29)
I don't hear about it that much in my peer group. I think it's because it doesn't affect people our age quite as much. I looked it up, and when you're in your 50s, only about one in 17 people will get shingles.
 
[Kristin] (7:30 - 7:50)
Yeah, it gets much more common as you age because your immunity goes down. And most people only get it once, but you can actually get it more than once. But things are different for people born starting in the mid-1990s, Regina, and that's because most of them got the chickenpox vaccine, so they never got chickenpox.
 
Like, my kids never had chickenpox. They were vaccinated.
 
[Regina] (7:51 - 7:55)
Okay, so the people really at risk of shingles is us, Generation X.
 
[Kristin] (7:56 - 8:48)
Yep, people who got the chickenpox vaccine, the generations after us, they have a much lower risk of shingles.
 
[Regina]
Do they get off scot-free? They can never get shingles?
 
[Kristin]
No, they aren't at zero risk of shingles because the chickenpox vaccine is actually a live, attenuated vaccine. It has real virus in it. So, technically, it's possible that that weakened virus could take root in your nerve cells and cause shingles later, but it's much, much less likely.
 
The good news for our generation, though, Regina, is we do have a vaccine against shingles. The first vaccine was Zostavax, and that came out in 2006. That was also a live, attenuated vaccine, and it gave some protection against shingles, but it wasn't terrific, and the immunity waned over time.
 
So, it's no longer used in the U.S. It was discontinued in 2020.
 
[Regina] (8:48 - 8:55)
The one I've heard of is Shingrix, though, which, by the way, also sounds like a Dr. Seuss character. Does it not?
 
[Kristin] (8:55 - 9:13)
Totally, yeah. Yeah, we've had Shingrix since 2017, and that is a recombinant vaccine, meaning it's not a live virus. It's just a viral protein.
 
You do need two doses, and it's recommended for anyone over 50 as well as those who are immunocompromised.
 
[Regina] (9:14 - 9:21)
But what about the people who got the old vaccine before 2017? Are they supposed to get the new one, too, on top of it?
 
[Kristin] (9:21 - 9:48)
Yeah, they are. Actually, the CDC recommends that they get Shingrix even though they had the old vaccine because Shingrix gives so much better protection. It's very effective, something like 97% efficacy in healthy adults aged 50 to 69, and even in those 70 and over, the efficacy is still like 90%.
 
So, Regina, you and I are both admittedly over 50. Have you gotten your Shingrix vaccine yet?
 
[Regina] (9:49 - 9:51)
Not yet. I confess, not yet. Maybe after this episode.
 
[Kristin] (9:51 - 10:19)
Maybe. I actually went out and got my Shingles vaccine, both doses, recently.
 
I decided actually to go on a vaccine binge recently because I was feeling a little nervous about the national climate right now and the availability of vaccines in the future. And so I figured I would just go out and get up to date on everything I was due for. So I got pneumococcal, Hep B, flu, COVID, and Shingrix.
 
[Regina] (10:20 - 10:28)
I've heard that Shingrix really knocks you out, like for a long time. That's why I have not gotten mine yet. Did you get any side effects?
 
[Kristin] (10:29 - 10:34)
Regina, I hate to admit it, but I actually did get side effects from the Shingrix vaccine.
 
[Regina] (10:34 - 10:41)
Whoa. Is this finally a sign that you're human, Kristin? Because you tend to be, let's say, exceedingly healthy and robust.
 
[Kristin] (10:41 - 11:03)
Well, minus the cancer, but yes.
 
[Regina]
Okay. Yeah.
 
[Kristin]
You know, this is weird. This is the first vaccine I have ever had a reaction to. And my doctor had warned me that it might cause side effects.
 
And I was like, no, vaccines never bother me. I've never had a problem. But I did, with the Shingrix vaccine, actually get fever and chills with both doses.
 
[Regina] (11:04 - 11:15)
So I'm going to be out like a week then. If they hit you at all, I need to prepare. So is there a high uptake in the U.S., or are there a lot of people like me out there avoiding it?
 
[Kristin] (11:15 - 11:48)
Actually, most people are like you, Regina, avoiding it. Chicken.
 
[Regina]
Chicken.
 
Chicken for chickenpox.
 
[Kristin]
Yes, exactly. Yeah.
 
According to national data, only about one-third of adults over 50 have had any shingles vaccine, including the old one. And just 18 percent, less than a fifth, have completed the recommended two-dose Shingrix series. So most of the people who should have gotten it actually haven't.
 
And I am in the minority and actually feeling quite virtuous now.
 
[Regina] (11:49 - 12:10)
You should be feeling virtuous about this. You survived all of this. And I'm feeling like I really need to go out and just make the time to be knocked out like this.
 
But Kristin, we are here to talk about dementia, not just about shingles. Why would this shot potentially reduce the risk of dementia? Like what is the biology here?
 
[Kristin] (12:11 - 12:47)
Right. When I first saw these headlines, I was like, wait, what's the connection between shingles and dementia? But here it is.
 
The idea is that the shingles virus lives inside nerve cells. And if it reawakens, it can cause inflammation in those nerve cells, including potentially neurons in the brain. And inflammation may help lead to dementia.
 
[Regina]
Oh, interesting.
 
[Kristin]
Yeah. Here's another interesting fact, Regina.
 
Varicella zoster, the virus that causes shingles and chickenpox, it's actually a virus in the herpes virus family.
 
[Regina] (12:49 - 13:04)
Perfect. So we are legitimately getting sex into this episode, right? Thank you.
 
[Kristin]
You're welcome.
 
[Regina]
Just to be clear, we are talking about the same family that includes the virus that causes genital herpes. Yes?
 
[Kristin] (13:05 - 13:18)
Yes, exactly. Yes. It's a family of viruses that includes an STD, but also other non-STD diseases.
 
Yeah. So this family of virus, oh, it also includes Epstein-Barr virus, Regina.
 
[Regina] (13:19 - 13:33)
Oh, which is actually another connection to sex, right? Because that, if I remember correctly, is the virus that gives you mononucleosis, sometimes known as the kissing disease, because it spreads through saliva, if I remember.
 
[Kristin] (13:33 - 13:34)
That's right.
 
[Regina] (13:34 - 13:42)
So we are hitting all of the glamorous parts of sex today. Right.
 
[Kristin]
Regina, did you ever have mono growing up?
 
[Regina]
I don't think I did, actually.
 
[Kristin] (13:42 - 13:49)
I didn't either. Clearly, neither of us was doing enough kissing in high school.
 
[Regina] (13:49 - 13:53)
We need to make up for lost time right now.
 
[Kristin] (13:53 - 14:26)
Absolutely. All right. Quick biology lesson here, Regina.
 
There are nine human herpes viruses in this herpes virus family. So these include the virus that causes cold sores, the one that causes genital herpes, the varicella zoster that we're talking about today, Epstein-Barr virus, and five others. I'll put them all in the show notes.
 
And they all share one superpower. They can go latent. They can hide quietly inside your cells, often in nerve cells, and can reactivate later.
 
[Regina] (14:26 - 14:32)
Ooh. That is creepy. Kind of like my ex-boyfriends just showing up out of nowhere.
 
[Kristin] (14:33 - 14:40)
Right. Resurfacing later. And they can be painful when they resurface.
 
So it is a little like shingles, Regina.
 
[Regina] (14:40 - 14:45)
And it's kind of ugly.
 
A little blistery. Can be. Can be.
 
Yes.
 
[Kristin] (14:45 - 15:23)
With some of your exes, yes. I know. I like some of your exes.
 
I'm not, yeah, panning them all.
 
Okay. We've wandered here.
 
[Regina]
We’ve wandered. All right. Back to the theory of how we get into dementia here.
 
[Kristin]
So the theory goes like this. Viruses in the herpes family can lie dormant for decades, and they occasionally flare up, and they can cause inflammation, and that inflammation may injure neurons in the brain, and that could increase your risk of dementia.
 
[Regina] (15:23 - 15:29)
Hmm. So is this just theoretical speculation, or do we have any evidence of any of this?
 
[Kristin] (15:29 - 15:57)
We have some circumstantial evidence, Regina. It is biologically plausible. The best evidence we have so far actually comes from the Epstein-Barr virus, the mono virus, because that virus has been strongly linked to multiple sclerosis, which, of course, is a neurological disease.
 
And, Regina, I'm going to put some more information about the links between herpes family viruses and neurological diseases in the show notes for anyone who's curious.
 
[Regina] (15:57 - 16:22)
That is a really fascinating link, right? So it's not just this chickenpox shingles dementia, but it might be part of the bigger trend. Okay, so the idea is if you get the shingles vaccine, then you most likely won't get shingles, so that would mean you avoid that particular reactivation and inflammation that goes with it, and so you end up lowering your dementia risk.
 
Is that it?
 
[Kristin] (16:22 - 17:16)
That is it, exactly. And this is probably not going to prevent all cases of dementia. Maybe this is just a small subset of causes of dementia, but if you avoid the shingles, maybe you're going to avoid some cases of dementia.
 
All right, so that's the background biology. And, Regina, before we get into that first paper, let's take a short break first.
 
Welcome back to Normal Curves.
 
Today we're examining the claim that the shingles vaccine prevents dementia. And, Regina, you were going to tell us about a paper in Nature that came out a few months ago.
 
[Regina] (17:16 - 17:29)
Kristin, I think you will like this one. For one thing, most of the research team is at Stanford.
 
[Kristin]
Ah.
 
[Regina]
Yeah. The paper came out in April 2025 and made quite a few headlines at the time.
 
[Kristin] (17:29 - 17:37)
Yeah, I remember. It was all over social media. And, Regina, this one focused on the old vaccine, right?
 
The Zostavax and not Shingrix, correct?
 
[Regina] (17:38 - 18:35)
Right, the old one. But it's got a really clever study design. Okay, let's pretend, Kristin, by way of illustration, you are 79 years old.
 
You are living in Wales, and it's 2013. And your 80th birthday is coming up on September 1st.
 
[Kristin]
Regina, I don't like this picture.
 
[Regina]
You'd make a great older Welsh woman.
 
What am I talking about? Okay, now, Wales is launching a new shingles vaccination program right around you.
 
But here is the kicker. Your neighbor also turns 80 this year. Their birthday is September 2nd, one day after yours.
 
And the government says, sorry, Kristin, you just missed the cutoff for this shingles vaccination. You will never be eligible for the vaccine in this program ever. But your neighbor, born one day later, is eligible.
 
[Kristin] (18:36 - 18:36)
Okay, that seems weird.
 
Why would they do it that way?
 
[Regina] (18:37 - 19:42)
I know. It's just how the government rolled it out. They had to draw the line somewhere, I'm guessing.
 
And when the program launched in 2013, they said, all right, anyone turning 80 on or after September 2nd gets the vaccine. Anyone before that doesn't. It's a one-day bright line difference.
 
That's it. So, Kristin, maybe harsh for you, but for investigators, it's fantastic because that bright line cutoff gives us what we call a natural experiment.
 
[Kristin]
Right.
 
[Regina]
Two groups that are basically identical except for this arbitrary rule. Right. The arbitrariness turned out to be a feature, not a bug.
 
It's a good thing because we didn't have to randomize anyone. The Welsh government basically did it for us because there's no reason to think people born on September 1st differ in any meaningful way from those born September 2nd. They're the same age, same country, everything.
 
It's just that only one group could get vaccinated, and that's our treatment in the study.
 
[Kristin] (19:43 - 20:01)
That is cool, Regina, because it fixes the big problem with most observational vaccine studies, which is healthy user bias. People who choose to get vaccines are often healthier. So, if you see lower dementia rates in them, you wouldn't know if it came from the vaccine versus their healthy lifestyle.
 
[Regina] (20:02 - 20:41)
The design gets around all that because they're basically assigning you to which group you're in by whether your mother went into labor before or after midnight, which is pretty random. So, it's not a randomized trial, but it's pretty close for the real world, and we really like these. So, the researchers were able to just follow both groups over time, over a seven-year follow-up period in the Wales healthcare system to see what happened.
 
And they asked, did those September 2nd babies who could get the vaccine end up with fewer dementia diagnoses than the ones who had just missed out?
 
[Kristin] (20:41 - 20:53)
Wait, so did they literally compare everyone born on September 2nd, 1933 to everyone born on September 1st, 1933? Because that cannot be a lot of people, Regina. Wales is not that big.
 
[Regina] (20:54 - 21:04)
Okay, good catch. You saw my kind of fake-out setup here. It's good for illustration, though, but you're right.
 
If you did that, you'd end up with only about 50 people in each group.
 
[Kristin] (21:04 - 21:10)
Oh, wow. Yeah, that would be really tiny, and you would barely have any dementia cases. So, way too small to draw conclusions.
 
[Regina] (21:10 - 21:57)
Exactly, but that's actually the first challenge in this kind of study, and we call it a regression discontinuity design. Oh, fancy. Super fancy, RDD.
 
It's just, you've got this sharp dividing line, right? We need that. And in this case, it's the vaccine eligibility birthday, but the challenging question is, how far out from that line do we reach to include people in the analysis?
 
If you only look one day on each side of that line, sure, it's super clean, but it's just too small of a sample like we talked about. And I think it's worth kind of unpacking this for a moment, Kristin, because it's a great example of what statisticians are always going on about, the bias-variance tradeoff.
 
[Kristin] (21:58 - 22:03)
Oh, yes, bias-variance tradeoff. That is a tension that's at the heart of a lot of statistics, Regina.
 
[Regina] (22:03 - 23:16)
Yeah, and it sometimes is a little vague, but the idea is that we're balancing two goals. So, first, we want a clean comparison. And here, it's people born close together, so their only real difference is the vaccine.
 
But then, second, on the other hand, we need enough people to make the numbers stable. If your time window is too narrow, you get high variance. And by that, we mean the results are jumping around because you don't have enough data, like the data are noisy, that's what we say.
 
But if your time window is too wide, you're getting bias, what we call bias. And we mean here that the comparison stops being fair, that we can't really answer the research question fairly. So, the trick is finding that sweet spot.
 
And here, the statisticians are calling it optimal bandwidth, but Goldilocks' time window, let's just say that. Not too narrow, not too wide, just right.
 
[Kristin]
Yeah, and how did they choose that window here, Regina?
 
[Regina]
Well, with math, of course, a fancy algorithm involving calculus. And it's cool. It just mathematically balances those two sources of error that we were talking about, bias and variance.
 
And it finds that Goldilocks point automatically.
 
[Kristin] (23:17 - 23:22)
Ah, great. So, the algorithm finds our optimal time window. And then, what happens next?
 
[Regina] (23:22 - 24:22)
So, to illustrate this, Kristin, let's just pick a number to be concrete. Let's say the window spans eight years total. So, we'd have four years on each side of the cutoff.
 
And the next thing is that we can actually make this whole analysis even better with one clever trick. What's the clever trick? The clever trick is giving different weights to people in the sample.
 
So, we are going to make it so that not everyone is contributing equally to the results, which is a little weird, but it makes sense if you think about it. Because the people with the birthdays closest to the cutoff, let's say in this example, within one year on either side, they will get the highest weight because they're the ones who are giving us the cleanest, most trustworthy comparisons. But then as you move farther out, the weights will gradually fade at a steady rate.
 
So, those born in the second year out from the cutoff weighted at 75 percent, and then 50 percent in the third, 25 percent in the fourth, and then you go to zero.
 
[Kristin] (24:22 - 24:25)
So, you're kind of fading out the data as you move away from the cutoff.
 
[Regina] (24:26 - 24:52)
Yeah, yeah, it's very visual when you think of it that way. This is called triangular kernel weighting, which is, again, another jargony term. I'm thinking popcorn.
 
It had nothing to do with popcorn kernels. The reason is if you plot those weights, you get a triangle high in the center and then tapering to zero at the edges. It's really just a neat and elegant way to squeeze out the most information from the cleanest data but without just getting rid of the rest of it.
 
[Kristin] (24:52 - 25:00)
That's really cool. It's like giving more weight to advice from your closest friends but not ignoring other people in your life completely.
 
[Regina] (25:02 - 25:24)
I give your advice a little bit more weight than I do others.
 
[Kristin]
Likewise, Regina.
 
[Regina]
Okay, so they have the time window.
 
Now, we set up the weighting. The next step for the investigators, before we get to the dementia, was this really kind of crucial point that we need to check off. They needed to double-check that the shingles vaccine actually did what it's supposed to do, which is reduce shingles.
 
[Kristin] (25:25 - 25:33)
I love these kinds of gut checks. It's just to make sure that there's nothing funny going on in your data, that, you know, shingles vaccine does indeed prevent shingles as expected.
 
[Regina] (25:33 - 26:05)
Right. There you go. Because if it didn't, then we're just going to have to stop there.
 
Okay, so they calculated that optimal time window. It's about 117 weeks on each side of the cutoff, so a little over two years in either direction. They applied that triangular weight.
 
They had about 76,000 people in the analysis. Well, that's certainly better than just 100 that went out one day. Okay, here, Kristin, is where the discontinuity part of regression discontinuity design comes in.
 
It's actually kind of cool and pretty visual.
 
[Kristin] (26:05 - 26:07)
Good, I like visuals.
 
[Regina] (26:07 - 27:50)
So, I am going to do this fun thing where I am describing a graph to you without you seeing it. Okay, think of it this way. We are going to fit two regression lines, that's two best-fitting lines, one for the group of people that missed the vaccine cutoff, and another one for those who made the cutoff, so they were eligible for the vaccine.
 
Each line is showing how the risk of shingles changes with age. Right, because we'd expect that risk to increase as people get older. Exactly.
 
Okay, now picture these two lines. They're plotted on the same graph, and the x-axis, that's the horizontal axis, is age. We flipped it around, so we have older ages on the left and younger ages on the right, and right in the middle of that x-axis is that cutoff date for when the vaccine program started.
 
So, the line on the left is showing everyone who turned 80 before September 2nd, 2013, the ones who were never eligible for the vaccine, but that's a mouthful, I'm just going to call them control group. And the line on the right is showing everyone who turned 80 after that date, that they were eligible, we'll call them the vaccine group. Now, here is the key.
 
We ask, what happened right at that dividing line? If those two lines meet smoothly, right, there's no gap, there's no jump, it's just kind of one continuous straight line crossing over the cutoff, then we can say the vaccine program didn't seem to change anything. The risk of shingles just kept following the same trend across both sides.
 
It didn't matter whether you were in the vaccine program or not.
 
[Kristin] (27:51 - 27:54)
Right, so no jump in the lines means no vaccine effect.
 
[Regina] (27:54 - 28:20)
Right, but on the other hand, if there's a sudden drop at that birthday cutoff line in the middle, like a little cliff, like a drop from the line on the left to the line on the right, that's the discontinuity, it's discontinuous. And that downward jump means the vaccine program caused a real reduction in risk of shingles for the people just on the vaccine side.
 
[Kristin] (28:20 - 28:25)
So you're literally looking for a step down in the graph right at that birthday cutoff.
 
[Regina] (28:26 - 29:10)
Right, so the slopes might tilt a little bit differently on each side, that's fine. What matters is that they don't line up smoothly. That little break, that downward step, that is the vaccine effect.
 
And the bigger the cliff, the bigger the drop, the bigger the effect.
 
[Kristin]
Got it. And so did they see a cliff here?
 
[Regina]
They did. People born before the cutoff or control group, their risk of shingles over the seven-year follow-up period was about 6.2%. And those in the vaccine group, their risk was about 5.1%. So simply being eligible for the vaccine lowered shingles risk by about 1.1 percentage points or about one fewer case per 100 people.
 
[Kristin] (29:10 - 29:28)
And Regina, I just want to remind everyone, this was the old vaccine, which did not offer perfect protection. So that's what we're seeing here. And of course, this was just vaccine eligibility.
 
Not everybody on the right side of the line probably went out and got the vaccine. Do we know how many people who were eligible actually got it?
 
[Regina] (29:28 - 29:43)
We absolutely do. This is an excellent catch. Not everyone who is eligible for the vaccine got it.
 
Only about 47% of the eligible group actually got vaccinated, meaning more than half of our so-called vaccine group never actually got the vaccine.
 
[Kristin] (29:44 - 29:50)
So that's like running a clinical trial where half the people in the treatment group never took the drug. So lots of noncompliance.
 
[Regina] (29:51 - 30:04)
Right. And that's why, actually, Kristin, this version of a regression discontinuity design is called fuzzy regression discontinuity. And I don't know about you, but I'm picturing a Muppet. Fuzzy.
 
[Kristin] (30:05 - 30:06)
Fozzy Bear.
 
[Regina] (30:06 - 30:21)
Fozzy Bear. I loved Fozzy Bear. Wasn't he like the keyboardist who would jam out?
 
[Kristin]
I think so, yes. I think so. Okay.
 
[Regina]
I love Fozzy the Bear. So now I'm picturing Fozzy the Bear running a regression discontinuity design analysis.
 
[Kristin] (30:21 - 30:23)
Jumping over the cliff, yes.
 
[Regina] (30:24 - 30:37)
Right. Okay. Now, fuzziness means not everyone followed their assignment. But the fix is simple.
 
We just scale up that observed effect and pretend as if everyone had, in fact, gotten the vaccine.
 
[Kristin] (30:38 - 30:54)
Right. So if only half of the people got it, you would just double the observed benefit to estimate what full vaccination would do. And, Regina, it's kind of like what we talked about in the alcohol episode with Mendelian randomization.
 
You do that same kind of up-weighting.
 
[Regina] (30:54 - 31:45)
Exactly. So remember, being in the vaccine group reduced shingles risk by about 1.1 percentage points. Only 47 percent of people in the vaccine group got the shot.
 
We divide 1.1 by 0.47. So our estimated true effect of the vaccine is roughly a 2.3 percentage point drop in risk of shingles over the seven-year follow-up period. And you and I like to talk about baseline risks with natural numbers. So big picture, for every 100 people in our study who don't get the vaccine, about six would get shingles during that follow-up period.
 
Data suggests that if everyone gets the vaccine, it would drop to about four out of 100. So we're preventing just over two cases per 100 people vaccinated. And these numbers line up great with what clinical trials found.
 
[Kristin] (31:45 - 31:59)
That's great, because it means that this natural experiment approach is doing exactly what we'd hope, reproducing what randomized trials have already shown. But, Regina, that's the effect on shingles, which we, of course, already know about. What about dementia?
 
[Regina] (31:59 - 32:16)
What's the effect on dementia? Right. The main event, dementia.
 
So for this, I used the same stats approach we talked about earlier. And the optimal time window here was about 91 weeks on each side, so just under two years. And that gave them about 56,000 people total.
 
[Kristin] (32:16 - 32:20)
All right. So slightly smaller than with the shingles outcome, but still a pretty big sample.
 
[Regina] (32:21 - 32:52)
Right. And the results. They found that people in our control group, their risk of dementia over the seven-year follow-up period was about 17.5%, so quite high. And those in the vaccine group, their risk was about 15.8% over the follow-up. And once you adjust for the fact that not everyone got vaccinated, it translates to about a three-and-a-half percentage point reduction in dementia from the vaccine itself, according to the data.
 
[Kristin] (32:52 - 32:53)
That's actually a lot.
 
[Regina] (32:53 - 33:23)
Yeah. So baseline risks, again, in natural numbers, out of every 100 people in our study, who you have to remember are about 80 years old, about 17 or 18 of them would be diagnosed with dementia over that seven-year period. And the data suggests that if everyone were vaccinated, it would drop to about 14 out of 100.
 
So with that vaccination per 100 people, we could be protecting about three to four people from developing dementia.
 
[Kristin] (33:23 - 33:29)
And it is just a shot, relatively easy to do, so kind of a lot of bang for your buck, actually.
 
[Regina] (33:29 - 33:41)
Right. But notice something interesting, Kristin. When you're looking at this, you might think that the results say that the shingles vaccine seems to protect more people from dementia than from shingles.
 
[Kristin] (33:42 - 33:43)
Oh, yeah. Interesting.
 
[Regina] (33:43 - 34:00)
When I first saw that, I mean, maybe it's obvious, but it felt counterintuitive to me. Like, is the shingles vaccine secretly a better dementia vaccine than a shingles vaccine? Because that would be weird.
 
So do you want to think through that and unpack that for us, Kristin?
 
[Kristin] (34:01 - 34:16)
Yeah, well, it looks like from the numbers you just gave, a lot more of these people are getting dementia, I think you said 17 out of 100, than are getting shingles. I think you said 6 out of 100. So maybe there's just a lot more dementia than shingles in people in their 80s.
 
 
[Regina] (34:16 - 36:04)
Exactly.
 
The baseline risk of dementia in this population is higher. So that's why we can prevent more cases. But, Kristin, here's a good place for just a quick reminder.
 
When we talk about the benefits of vaccines, we actually have two ways mathematically that we can express that. So far, we've been talking about absolute risk reduction, which is how many fewer people get the disease in absolute terms. But we can also express the same results in terms of the relative risk reduction.
 
And when we do that here, the impact looks a little different because the relative risk reduction is actually bigger for shingles than for dementia. For shingles, the vaccine lowered risk by about 2.3 percentage points. And since the baseline risk was around 6%, that works out to roughly 37% reduction in relative terms.
 
And, Kristin, you and I like to use analogies with shopping. So let's say an item costs $6 and the price is dropped by $2.30. That's a 37% savings.
 
Now, for dementia, the vaccine appeared to lower risk by about 3.5 percentage points. But since the baseline risk was higher here, around 17%, that works out to just a 20% reduction in relative terms. Because, again, let's say something costs $17, the price is dropped by $3.50. So that's now only a 20% discount.
 
[Kristin] (36:04 - 36:10)
So the relative discount is bigger in the first case, but the second deal saves more money in absolute terms.
 
[Regina] (36:10 - 37:07)
Right. So it's important to look at both the absolute risk reduction and the relative risk reduction because they both give you different information. Now, Kristin, there's one other result I want to talk about from this paper.
 
They did an exploratory subgroup analysis that gave a really surprising result.
 
[Kristin]
Oh, tell me more.
 
[Regina]
OK.
 
It was an exploratory analysis where they looked at men and women separately. And get this, that protective effect of the vaccine on dementia that we just talked about, in men, it was basically zero.
 
[Kristin]
Oh, wow.
 
[Regina]
Zero effect, but a pretty strong effect in women. For women, the reduction was about 5.6 percentage points. That means for every hundred women vaccinated, the shingles vaccine could possibly prevent five or six cases of dementia over seven years.
 
[Kristin] (37:07 - 37:10)
Wow, that's big. And any idea why it would differ between men and women?
 
[Regina] (37:11 - 37:39)
Yeah, it was kind of unsatisfying. The researchers did not have a clear answer. They were very cautious about it.
 
They said, hey, it might be sample size because dementia is less common in men at that age. It might be because most of them are dead by that point. So, there might not have been enough cases to detect an effect, or it could be that men and women respond differently to vaccines, or they have different biological pathways to dementia.
 
So, it was interesting, but definitely exploratory.
 
[Kristin] (37:40 - 37:49)
It is interesting because the paper I'm going to talk about, Regina, also found a bigger effect in older women. So, there's at least some consistency across the two studies.
 
[Regina] (37:49 - 38:01)
Oh, that is good. I love it when results triangulate. I want to point out one more great thing that they did because, Kristin, I think you will like this.
 
They ran negative controls.
 
[Kristin] (38:02 - 38:05)
Oh, we talked about negative controls back in the Scary Bridge episode.
 
[Regina] (38:05 - 38:16)
Right. So, here they checked to see if the shingles vaccine protects against things it should not protect against, like stroke or diabetes or falling down.
 
[Kristin] (38:17 - 38:28)
Oh, very nice. I really like this because it means that there is specificity for dementia, which makes it more convincing that this isn't just some kind of artifact, like related to when people were born.
 
[Regina] (38:29 - 38:58)
Exactly. So, luckily, nothing showed up. We are good on this.
 
So, Kristin, even though this study doesn't prove causality, I'd call it a pretty solid flirtation with causality. Not as satisfying as a full-on, you know, randomization, but it's definitely giving us some pretty good vibes anyway. What it doesn't give us, though, is generalizability.
 
What about people younger than 80? What about the newer vaccine? How does all of that compare?
 
[Kristin] (38:58 - 39:06)
Regina, I'm glad you brought that up because I've got a giant longitudinal study that tackles exactly those questions that we can look at next.
 
[Regina] (39:06 - 39:48)
Oh, perfect. Can't wait. Good time for a short break.
 
Welcome back to Normal Curves. Today, we're examining the claim that Shingle's vaccine prevents dementia. And, Kristin, you are going to tell us about a recent paper that looked at the newer vaccine, Shingrix, as well as a wider range of ages, not just elderly people.
 
[Kristin] (39:48 - 40:10)
This paper came out in October of 2025, so very recent, and it got a lot of headlines as well. Regina, unlike your study, which took advantage of a natural experiment, this study used a different approach to try to achieve balanced groups. It was also based on electronic health records, but more recent records and also records from the United States.
 
[Regina] (40:10 - 40:20)
That's more tricky than it is in Wales, because Wales has a national medical record system and the U.S. does not. So, what kind of data did they use?
 
[Kristin] (40:20 - 40:23)
Regina, they used the Optum database. Have you heard of that?
 
[Regina] (40:23 - 40:33)
I do keep hearing this Optum database. It sounds like it's a magical fountain of health data and hot chocolate. So, who exactly is Optum?
 
[Kristin] (40:33 - 40:51)
Yeah, it's basically a giant data warehouse, and it's run by an insurance company. They partner with hospitals and clinics all over the country to collect electronic health record data, and they clean, standardize, and anonymize the data and package it up for research.
 
[Regina] (40:52 - 40:59)
Giant warehouse of data. Why am I picturing like a Costco? Like a Costco of medical records.
 
[Kristin] (41:00 - 41:33)
Yeah, it's kind of like that. The researchers come along with a cart and they pick out the data they need. It's a massive data set.
 
Health records from over 100 million unique people seen at more than 7,000 hospitals and clinics across the U.S. And the data start in about 2007, because that's when electronic medical records started to come online in the U.S. And because it's longitudinal, meaning collected over time, we can reconstruct timelines for each patient. Of course, it's de-identified, so no one can be personally identified.
 
[Regina] (41:33 - 41:44)
So basically, Kristin, you're saying that now, instead of researchers needing to sign people up for a study, they can just sit down with a monster spreadsheet that they picked up at the Costco data warehouse?
 
[Kristin] (41:45 - 41:58)
That's exactly right, and it's incredibly powerful. I do have to give the caveats, of course, that with any massive, complicated data set like this, there are bound to be some errors. The data quality may be variable, so we always have to keep that in the back of our minds.
 
[Regina] (41:58 - 42:04)
Okay, so who do they look at in this Optum database? Who do they put in their Costco data cart?
 
[Kristin] (42:04 - 42:23)
They selected people 50 and older, because that's when shingles starts showing up and when people are eligible for the vaccine. And they looked for people with different exposures, such as getting the Shingrix vaccine but also getting shingles. And they made sure that at the time of that exposure, the person did not yet have a dementia diagnosis.
 
[Regina] (42:24 - 42:34)
Right. Exposure needs to come before the outcome. A vaccine cannot prevent dementia if you already have dementia.
 
So you mentioned a lot of exposures. What did they look at?
 
[Kristin] (42:34 - 43:41)
They looked at whether someone had had shingles, whether they got the old shingles vaccine, whether they got Shingrix. They made seven different comparisons by building seven different what we call matched cohorts. For example, they compared people who had multiple bouts of shingles to people who had shingles only once.
 
And the idea there is if shingles is related to dementia, then you'd expect the people who had had multiple episodes to be more likely to develop dementia.
 
[Regina]
Oh, that makes sense, like a dose-response effect.
 
[Kristin]
Exactly.
 
They also compared people who got the Shingrix vaccine to people who had only gotten the old vaccine. And remember, Shingrix is more effective. So the idea there is if you have more protection against shingles from the better vaccine, then you should get less dementia.
 
And they compared people who got the full two-dose Shingrix series to people who got a control vaccine that was unrelated to shingles.
 
[Regina]
Oh, that's interesting. Which vaccine?
 
[Kristin]
They used the pneumococcal vaccine, which is a vaccine that prevents bacterial disease such as pneumonia and meningitis. And it's not believed to have any connection to dementia.
 
[Regina] (43:41 - 43:54)
Ah, so it's a good control because when you pull in people who chose to get the pneumococcal vaccine, you're getting all those people who are health-conscious and willing to get vaccines in general. So it's a better comparison.
 
[Kristin] (43:55 - 44:27)
Exactly. And Regina, I want to go through the methods of the paper now, but I'm going to focus on just one of those seven comparisons because the methods for all of the comparisons are the same. So let's just focus on this comparison between the Shingrix vaccine and the pneumococcal vaccine. So what they did is they went through the Optum records and they found people 50 and up who had either received two doses of Shingrix, but not the pneumococcal vaccine, or had received the pneumococcal vaccine, but not Shingrix.
 
And of course, they were all dementia-free at the time of vaccination.
 
[Regina] (44:27 - 44:40)
Right, so just one or the other vaccine, not both vaccines, because if you're including people with both, that would just muddy the waters. So I've had a pneumococcal, but not shingles, so I'd be in the pneumococcal group.
 
[Kristin] (44:40 - 44:43)
Oh, that's interesting. Why did you get the pneumococcal vaccine?
 
[Regina] (44:43 - 44:47)
Oh, for cochlear implant. It's recommended for people with cochlear implant.
 
[Kristin] (44:47 - 45:02)
Oh, interesting. Yeah, so you would be in the pneumococcal group. I, on the other hand, have had both, so I would be excluded from this analysis.
 
They found about 670,000 people for the Shingrix cohort and about 4.3 million for the pneumococcal cohort.
 
[Regina] (45:03 - 45:10)
Ah, a lot more for pneumococcal. Maybe because Shingrix hasn't been out as long? Only since what you said, 2017?
 
[Kristin] (45:11 - 45:40)
Yeah, that's it exactly. There's a lot more people who have had pneumococcal vaccines just because it's been around longer. Now, here's the problem.
 
It turns out that even though we have two groups who were both willing to get preventive vaccines, so they're alike in that, they are still really different groups, Regina. Remember, it's not randomized. It's observational.
 
And the Shingrix group was much younger on average. The Shingrix group was also much more likely to be on steroids and taking antiviral medicines.
 
[Regina] (45:40 - 45:49)
Hmm, so maybe the ones on Shingrix were immunocompromised because Shingrix vaccines were recommended for people who have compromised immunity?
 
[Kristin] (45:50 - 46:01)
Exactly. Also, the pneumococcal group, they had more chronic health problems, such as heart disease and diabetes, and they appeared more frail. But the Shingrix group, on the other hand, they were going to the doctor more often.
 
[Regina] (46:01 - 46:32)
Hmm, so maybe more health-conscious people who do a lot of preventive care, they're getting Shingrix, whereas the pneumococcal vaccine, it's given to more sicker patients, maybe when they're getting inpatient care, like in a hospital?
 
[Kristin]
Yeah, that's it exactly, actually.
 
[Regina]
So how do they compare these two groups when they are so different?
 
Since these differences we're talking about, Kristin, are almost certainly related to dementia. Older age and heart disease, they're related to dementia. So how are they comparing these different groups?
 
[Kristin] (46:32 - 46:41)
Yeah, this is the problem. So, Regina, this is where those causal inference methods come in, and the one we're going to talk about now is called propensity scores.
 
[Regina] (46:41 - 46:49)
Propensity scores, interesting, but also confusing. So I'm thinking we take a little bit of time to talk about what those really mean.
 
[Kristin] (46:50 - 46:52)
Yeah, let's take a statistical detour on propensity scores.
 
[Regina] (46:53 - 47:10)
Kristin, you know, I actually have an analogy in mind for this, and it's very chaste. There's no sex in it.
 
[Kristin]
Oh, okay.
 
[Regina]
I think it might work anyway. All right. So let me give it a go.
 
Let me just try it out. Remember the Barbenheimer summer from a few years ago?
 
[Kristin] (47:11 - 47:11)
Yeah.
 
[Regina] (47:11 - 47:34)
It was the Barbie movie and Oppenheimer, and they were both playing at the same time, and everyone was talking about which one was better. So I actually saw both in theaters, and let me tell you, the audiences could not have been more different. Barbie was all pink and teenage girls, and Oppenheimer was like older men from engineering.
 
So you see where I'm going with this?
 
[Kristin] (47:35 - 48:19)
Oh, yes. I like this analogy, Regina. So if you wanted to compare the ratings of the two movies given by these two very different audiences, that would be a biased comparison, right?
 
I mean, ideally, if you could, you'd want to randomize people as they walked into the theater, send half to Barbie, send half to Oppenheimer, and then you'd be able to fairly compare their ratings. But probably you're not really going to be able to do that, and you'd just have to use the audiences that actually showed up. So imagine that we were trying to do a study, and we had 500 people who saw Barbie and 500 people who saw Oppenheimer, and we wanted to compare their ratings of the two movies, but we know it's not a fair comparison because those audiences are wildly different.
 
[Regina] (48:19 - 48:23)
Right, and this is where I am thinking propensity scores come in, right here.
 
[Kristin] (48:23 - 49:01)
Yes. So here's the big idea. We can take all 1,000 people that we have data on.
 
We already know which group they are in. We know whether they saw Barbie or Oppenheimer, but we can fit a statistical model where the outcome of the model is which of those two groups they're in, which movie they saw. And we're going to throw into that model a bunch of predictors that capture the essence of who is likely to be in the Barbie group versus the Oppenheimer group.
 
So we might include predictors like age, sex, social media use, maybe whether they listen to NPR or read The Economist, and maybe some kind of score on a nerdiness questionnaire that we will give them.
 
[Regina] (49:02 - 49:18)
Whether they have the Barbie dream house and the Barbie convertible at home.
 
[Kristin] (49:19 - 49:59)
Ah, yes. Okay, so basically you're modeling who seems like they're a Barbie person or seems like they're an Oppenheimer person regardless of what movie they actually went to.
 
Now, this may seem really strange. Like, why are we building a model to predict who will see which movie when we already know who saw each movie? But the reason we are doing this is it allows us to calculate for each person a probability or propensity that they would be in the Barbie group as opposed to Oppenheimer.
 
And as we'll see, we can use these propensity scores to help balance the groups. So, for example, a teenage girl who loves social media, if we put her data in the model, maybe it'll spit out that she has, like, an 80% chance of seeing the Barbie movie. But a 60-year-old male engineer might have just a 15% chance of being in the Barbie group.
 
[Regina] (50:00 - 50:14)
Right, so everyone gets their own personal probability or propensity of being in the Barbie theater group. But, Kristin, now we have to talk about what we are doing with those probabilities because that is the fun part.
 
[Kristin] (50:14 - 50:50)
So once we have those probabilities, those propensity scores, we take everyone in the Barbie group and we try to find a match for them in the Oppenheimer group who has the same propensity score. So if someone in the Barbie group had a 30% chance of seeing Barbie, we look for someone in the Oppenheimer group who also has a 30% chance of seeing Barbie. And depending on how close we want those matches to be, we may have to drop some people from the dataset.
 
For example, maybe we can only find good matches for 250 people from the Barbie group, so we end up with 250 matched pairs and we drop the rest of the dataset.
 
[Regina] (50:51 - 50:59)
Hmm, so we are making matches scientifically. Not quite Tinder. I keep trying to think if we can bring dating back into this and it's not working, but we are making matches.
 
[Kristin] (51:00 - 51:48)
Yes, we're making matches. But here's the interesting part. We're used to thinking of matching in studies as finding someone who's basically a twin of another person.
 
Like we often match on exact age and sex and location. But in the case of propensity scores, that's actually not what's happening. At an individual level, the matched pair might be two people who are completely different.
 
So, for example, maybe we have a very nerdy teenage girl in the Barbie group and her propensity of seeing Barbie is 60%, and we end up matching her to a 40-year-old male in the Oppenheimer group who also has a 60% chance of watching Barbie, maybe because he loves social media and isn't that nerdy. According to the model, they're a perfect match. They both have a 60% probability, but they look completely different as individuals.
 
[Regina] (51:49 - 52:03)
Which is fascinating to me because we are matching just on that number at the end, right? The propensity score. And we don't really care so much about the reasons why you have that propensity score.
 
[Kristin] (52:03 - 52:20)
Exactly. But even though those individual pairs may not look similar, what this does overall is it balances the confounders on average across the two groups. So it's kind of a statistical magic trick, and it balances everything so we can more fairly compare the movie ratings from the two groups.
 
[Regina] (52:21 - 52:35)
Okay, let's get back to the study now, Kristin, not what happened in Barbie and Oppenheimer, although I'm kind of invested in this now, and I want to see which of the movies won. But yeah, okay, back to Shingles vaccine. What did they do here?
 
[Kristin] (52:36 - 54:05)
Right, so in this Nature Medicine study, they had 670,000 people in the Shingrix group and about 4.3 million in the Pneumococcal group to start with. And they threw all of this data into this sophisticated machine learning model with nearly 400 predictors. And the great thing about the machine learning model is it allows us to look at nonlinear relationships and higher-order interactions between the predictors.
 
You actually need a lot of data to be able to do that, but they certainly had plenty of data here. So they made this model, and then they calculated a propensity score, a probability of being in the Shingrix group for everybody, and then they went through, and for every person in the Shingrix group, they tried to find a match in the Pneumococcal group. And they tried to match very closely.
 
So if someone in the Shingrix group had a 20.2% probability, they looked for someone in the Pneumococcal group with also a 20.2% probability. And because they wanted those matches to be so close, though, they actually ended up throwing out a lot of people. Remember, they started with 670,000 in the Shingrix group.
 
They ended up, though, with just 234,000 matched pairs. So there were a lot of people thrown out who did not have a good match. They did check the two groups to see if they were balanced, and indeed, now the two groups had the same average age, same rates of heart disease, same number of people taking antivirals, same rates of visiting the doctor, and so on.
 
So they were now very balanced on those almost 400 confounders.
 
[Regina] (54:05 - 54:18)
Oh, great. So this really answers that question I posed to you a little while ago, Kristin. You have these two very different groups.
 
We're trying to compare them as if they were similar. How do you do that?
Propensity score matching.
 
[Kristin] (54:18 - 54:32)
Yeah, and Regina, we do have to keep in mind, though, this is still not as good as randomization because the groups are only balanced on confounders we could measure and confounders that we put in the model. So there may still be some unmeasured and residual leftover confounding here.
 
[Regina] (54:33 - 54:45)
Right, like we don't know how often they're doing the New York Times crossword puzzle to help their brain.
 
[Kristin]
That's right, yeah. We don't have that information, correct.
 
[Regina]
Okay, so now we have these two groups that look similar. How are we comparing them?
 
[Kristin] (54:45 - 55:30)
Right, so what we are doing is following the two groups forward in time. Everyone has an index date. That's the date they got the vaccine.
 
And then we track them over time in the medical records to see who went on to develop dementia. And people were followed up for about five years after vaccination. Now, Regina, not everyone could be followed for the entire five years because some people died or we lost them out of the medical records.
 
And those people are what we call censored. They stopped contributing data at that point. We also have a lot of censoring in this study because if somebody went on to get the vaccine in the other group, like somebody in the pneumococcal group went out and got the Shingrix vaccine, the researchers censored them at that point because they were no longer in the group of interest.
 
[Regina] (55:31 - 55:53)
Well, that makes sense because they want to keep everything nice and clean. And we talked about censoring back in our episode on exercising cancer. And we talked there about how it's just a little strange to use this word censor.
 
Right. We're censoring them statistically, to be clear. We are not censoring them because they said something wrong.
 
We're not bleeping them out.
 
[Kristin] (55:53 - 56:43)
That's right. We're just only using their data up to that point. Now, what was interesting to me, Regina, is the paper actually in the figures, they show how many people were censored in each group at every year post-vaccination.
 
And what I noticed is that even though the two cohorts are nicely matched, the patterns of censoring differed a lot between the groups. And this was true of all the different comparisons. And this is just a good reminder that these are not randomized groups.
 
If it were a randomized trial, you'd expect to see pretty similar patterns of censoring.
 
[Regina]
What kind of pattern did you see here?
[Kristin]
In the Shingrix versus Pneumococcal comparison, there was a lot more censoring in the Pneumococcal group.
 
And I'm wondering if that's because the Shingrix vaccine is new and maybe a lot of people who were in the Pneumococcal group went out and got Shingrix when it became available, but there were less people going in the other direction.
 
[Regina] (56:43 - 56:48)
Yeah, that would explain it, Kristin. But as you're talking, I'm thinking about how much data we're throwing away here.
 
[Kristin] (56:49 - 56:49)
Yeah.
 
[Regina] (56:49 - 57:00)
Because we started with millions in the cohort and now we're down to like 234,000 pairs. So, great, we got the balancing, but can we really generalize now? Because we got rid of so much data.
 
[Kristin] (57:00 - 57:34)
Yeah, absolutely. We have to worry about this selection bias because of who we're throwing out. And actually, to the author's credit, they did do a second analysis that's related to propensity scores but doesn't throw anybody out.
 
And it's called Inverse Probability of Treatment Weighting, or IPTW.
 
[Regina]
Horrible name. Such a horrible name.
 
[Kristin]
Yeah, I feel like somebody makes up this jargon just for job security, right? Like, we're going to make it sound really tough and complicated and something you can't remember or understand so you don't come and take my job.
 
[Regina] (57:35 - 57:42)
I feel like we need to counter that, Kristin, by coming up with names that have the potential to go viral on TikTok.
 
[Kristin] (57:43 - 57:52)
Ooh, I like that. Yeah, as we've said before, Regina, you and I just need to rename all of these jargony terms so that more people can understand these important topics.
 
[Regina] (57:53 - 58:09)
Right, a big tent. Invite everyone in. Okay, I'm going to take this as a challenge.
 
Kristin, while you are explaining this, I am going to come up with an alternative name for IPTW, and it's going to be better than IPTW.
 
[Kristin] (58:09 - 59:28)
Ooh, I love it. Great challenge. Okay, so let me explain it, and then we'll hear what you came up with. What this analysis does is it uses the same propensity scores that we've already calculated, so it's kind of a twofer. You've already got them hanging around.
 
You might as well do this other analysis while you're at it. Instead of matching and discarding people, it weights everyone by the inverse of their probability of being in their group. And let me explain that using the Barbie and Oppenheimer example again.
 
So suppose there was a dad who went with his teenage daughter to go see Barbie, and maybe based on his characteristics, his probability of being in the Barbie group was only 25%. In IPTW, we take the inverse or reciprocal of that probability, so 1 divided by 25%, that equals 4, and that means that we would weight his opinion four times. We would count him four times.
 
But let's say over in Oppenheimer there was someone who only had a 20% chance of seeing Oppenheimer. Maybe a teenage girl ended up over there somehow. So she would get a weight of 1 divided by 0.2 or a weight of 5. So basically, Regina, we are up-weighting the statistical oddballs, the people whose characteristics made them unlikely to end up in their actual group. And interestingly, this achieves the same balance between the two groups, but without dropping anyone from the sample.
 
[Regina] (59:28 - 59:35)
Hmm. It's just that some people are counted more than others, weighted more heavily than others, a little like we talked about in the Nature paper.
 
[Kristin] (59:35 - 1:00:21)
Exactly. And they applied this IPTW in the Nature Medicine paper as a second analysis. So the unexpected people in the Shingrix group got more weight in the Shingrix group, and the unexpected people in the Pneumococcal group got more weight in the Pneumococcal group.
 
So, for example, Regina, you are in the Pneumococcal group, but if we put your data into that fancy machine learning model, you would probably actually have a high probability of being in the Shingrix group. Right? Because you're younger, you do a lot of preventive care, and I don't think that they included cochlear implants as part of their statistical model.
 
So maybe you would have a 75% chance of being in the Shingrix group and only a 25% chance of being in the Pneumococcal group, and that means that your data would be weighted four times, fourfold.
 
[Regina] (1:00:22 - 1:00:27)
I love it. There's a benefit to being a statistical oddball, a benefit to being a misfit.
 
[Kristin] (1:00:28 - 1:00:43)
Exactly. And, again, this balances the two samples because, Regina, you're counted four times in the Pneumococcal group, and you're younger and you do more preventive care, so you make that group look more similar to the Shingrix group, and we end up achieving balance.
 
[Regina] (1:00:43 - 1:00:53)
You guys feel a little backwards. If you are unlikely to be in a group, we're going to upweight you. We're going to give you more voice if you're a misfit.
 
The oddballs are getting more attention.
 
[Kristin] (1:00:54 - 1:01:08)
That's exactly right. Now, in this study, they did do a little bit of trimming, so if you had a probability of like 1%, that would have given you a weight of like 100, and so they trimmed people at the very extremes just so nobody was overly influential in the data.
 
[Regina] (1:01:08 - 1:01:28)
Kristin, this is great. And now I think it's giving me a handle on the name, and maybe the new name should be something around statistical misfit or statistical oddballs because that's really what's happening here, but the problem is, Kristin, it doesn't have a nice acronym, Statistical Oddball Approach.
 
[Kristin] (1:01:29 - 1:01:30)
SOA.
 
[Regina] (1:01:30 - 1:01:44)
SOA, yeah. That's not going to be memorable. Kristin, this is actually a perfect task for LLMs. Oh. I don't know if you've ever used LLMs, large language models, for anything as silly as this.
 
[Kristin] (1:01:45 - 1:01:51)
Oh, that is a great idea. Get the LLM to come up with a name that has a really cool acronym. I like it.
 
Did you try it?
 
[Regina] (1:01:52 - 1:02:17)
Kristin, I confess, I have been doing a teeny bit of multitasking while we were doing this last section. I was listening to you. You were riveting, fascinating, gripping, so I was paying attention the whole time, but on the side, I asked my LLM, hey, can you come up with something to name this IPTW that's a little bit spicier and saucier and fits with the Normal Curves podcast, and it gave me some options.
 
[Kristin] (1:02:17 - 1:02:18)
Okay, what did you come up with?
 
[Regina] (1:02:18 - 1:02:30)
Okay, for this one, you've got to listen carefully because it's the acronym that's really exciting. Statistical Propensity Adjustment for Normalized K-Groups.
 
[Kristin] (1:02:30 - 1:02:34)
Okay. That actual name is not really better than Inverse Probability of Treatment Weighting.
 
[Regina] (1:02:34 - 1:02:39)
Yeah, I know. I told her to optimize the acronym, so the acronym S-P-A-N-K.
 
[Kristin] (1:02:40 - 1:02:46)
SPANK. SPANK is a good acronym, and people will remember that better than IPTW.
 
[Regina] (1:02:46 - 1:02:54)
Thank you, yes.
 
I would be way more likely to use this method if I was able to say, like on my poster, you know, in my talk, I SPANKed the data.
 
[Kristin] (1:02:55 - 1:02:57)
Okay, that is pretty good. Right? All right, what else have you got?
 
[Regina] (1:02:58 - 1:03:14)
Okay, this one is a bit of a reach, but you'll see that it's worth it at the end. Outlier Regeneration and Sample Magnification. O-R-G-A-S-M.
 
[Kristin] (1:03:16 - 1:03:22)
That is a great acronym. And sample magnification, it actually is reflecting what the method does. Not bad.
 
[Regina] (1:03:22 - 1:03:40)
I know. I think that LLM actually understood a bit about IPTW, but more importantly, it understood me. Orgasm.
 
Like, come on. Like, someone could say, hey, what did you do for your dissertation? And you could say, a lot of orgasms.
 
[Kristin] (1:03:42 - 1:03:43)
That is great. I love it.
 
[Regina] (1:03:44 - 1:04:12)
Okay, that one's a bit unwieldy, though. It takes up too many words. In the abstract, this next one is short and sweet.
 
It gets right to the point, unlike the orgasm, which might take time, you know. So, this one, it plays a little loose with the acronym part, but we're going to do it. Okay.
 
Statistical Exception  eXpansion. So, you got to lean on that X at the end. S-E-X.
 
[Kristin] (1:04:12 - 1:04:25)
S-E-X. I love it. And actually, Statistical Exception Expansion is a pretty good representation of what this method does.
 
Perfect. I think we've successfully renamed IPTW, Regina. Yeah.
 
[Regina] (1:04:26 - 1:04:27)
You can't go wrong with sex.
 
[Kristin] (1:04:27 - 1:04:30)
Absolutely. We've made an important contribution here, Regina.
 
[Regina] (1:04:31 - 1:04:39)
We have. Okay. Sadly, moving on from the sex.
 
Let's talk about the results. What did they get from this sex analysis?
 
[Kristin] (1:04:40 - 1:05:25)
Regina, the two different analyses actually got similar results, so I'm just going to focus on the main analysis, which was the propensity score matching. They made seven different comparisons. All of these comparisons showed significant benefits for dementia in the expected direction, with more shingles protection or less shingles giving less dementia.
 
[Regina]
Interesting. How big were the effects?
 
[Kristin]
In terms of the relative risk reduction, the effects were similar in size to what was reported in the Nature study that you just talked about, Regina.
 
So, to give an example, for Shingrix versus pneumococcal, the comparison we've been focusing on, there was a 27% relative reduction in dementia at three years and a 17% reduction at five years.
 
[Regina] (1:05:25 - 1:05:30)
Yeah, that's very similar to the 20% relative risk reduction we saw in that natural experiment.
 
[Kristin] (1:05:31 - 1:06:16)
Right. But the absolute risk reduction in this study was much smaller because, remember, this study looked at a much broader age range, lots of people in their 50s and 60s, so the absolute risk of dementia was less, and the Shingrix group had only about two or three fewer cases of dementia per 1,000 people. That's an order of magnitude less than the other paper.
 
Yeah, and again, that's likely due to the fact that this paper looked at younger age groups because they did do an exploratory analysis where they looked specifically at women 80 to 89, and in those women, the reduction in risk in the vaccinated group was about two and a half fewer cases of dementia per 100 people over a five-year period.
 
[Regina] (1:06:16 - 1:06:32)
Oh, yeah, that's actually pretty similar to what we talked about in the Nature paper. There, it was about five to six fewer cases per 100 people over seven years. I like how the results are kind of triangulating all together, right?
 
They're reinforcing each other.
 
[Kristin] (1:06:32 - 1:07:45)
Regina, there are a few quick caveats that I want to mention about this paper, and I'll put some more details in the show notes. First, with both propensity score matching or IPTW sex, we have to keep in mind that it's not as good as randomization. Both methods only account for measured confounding, and there still could be unmeasured or residual confounding.
 
Remember, we saw that the pattern of censoring was different in the groups, which is a good reminder that the groups might not be perfectly balanced. Now, to their credit, the authors did try to address this issue. They did some robustness checks, including something clever called a time-shifting analysis, basically pretending everyone got vaccinated a year earlier to see if differences in dementia showed up before vaccination.
 
Now, they concluded from this check that there wasn't much evidence of residual confounding, but I'm not sure that I agree with their interpretation of the results. So, for example, in the Shingrix versus Pneumococcal comparison, the Shingrix group still showed lower dementia rates, even in that fake time window, about the same 20% reduction that they reported in the main analysis.
 
[Regina] (1:07:46 - 1:07:51)
Kristin, doesn't that kind of suggest that the effect might actually be due to confounding?
 
[Kristin] (1:07:51 - 1:08:09)
Yeah, that's what I'm thinking. Now, I have to say, in this analysis, they only did a year of follow-up, and so the number of dementia cases was small, the confidence intervals were wide, so maybe we just can't draw many conclusions from this analysis, but it did not give me more confidence in the results. If anything, it made me more skeptical.
 
[Regina] (1:08:10 - 1:08:23)
So, overall, good study, impressive data, but not perfect. All right, I think we are ready to wrap up now. The claim for today is that the shingles vaccine reduces the risk of dementia later in life.
 
[Kristin] (1:08:24 - 1:08:41)
And how do we rate the strength of evidence for claims in this podcast with our smooch rating scale, one to five, where one smooch means little or no evidence for the claim, and five smooches means very strong evidence for the claim. So, Regina, how do you feel about this one? Kiss it or diss it?
 
[Regina] (1:08:41 - 1:09:22)
I'm going to kiss it. I'm going to smooch it all over. Four smooches. Four smooches, yeah.
 
I think what's convincing me here is the triangulation of the results from the different studies using different methods. Now, there are a lot of question marks here, like who does it apply to? Why are we finding zero effect in men and strong in women?
 
How long does the protective effect last? What age? But what I feel like is we might be onto something, that there might actually be a link between the shingles vaccine and dementia of some kind.
 
We don't know the mechanism, but there's something there. I'm just feeling it. And you?
 
[Kristin] (1:09:22 - 1:10:11)
Regina, I'm going to go just a hair below you with 3.5 smooches. I do find that natural experiment pretty convincing, and I do think the case is strengthened because the two papers have a lot of comparable results. So, I'm definitely more than half convinced.
 
I'm downgrading slightly from you to 3.5, though, because it isn't a randomized trial, and I think these causal inference methods can sometimes give us false confidence. I will recommend that everybody who's eligible should go out and get their Shingrix vaccine because we know that it prevents shingles. So, I think a few days of fever and chills from the vaccine is nothing compared to weeks of pain from a blistering rash.
 
And if, as a bonus, it also helps prevent dementia, I think it's definitely worth getting. Hint, hint, Regina.
 
[Regina] (1:10:13 - 1:10:37)
Yes, I feel called out here. This has convinced me. I will get a shingles vaccine.
 
I just need to set aside, like, a whole week of recovery afterwards.
 
[Kristin]
Oh, I don't think it'll take a week. It was only like a day for me.
 
[Regina]
I'll come convalesce at your house. How about that?
 
[Kristin]
Oh, yeah, well, Nibbles will take care of you.
 
[Regina]
Oh, the corgi nurse. That would be perfect. Okay, methodological morals, what do you have?
 
[Kristin] (1:10:38 - 1:10:46)
All right, here's mine, Regina. Propensity scores are the lipstick you put on observational pigs. I'm going a little cynical here.
 
[Regina] (1:10:47 - 1:10:54)
Very cynical. But it's got, like, a bit of a beauty thing in there, too. I like it.
 
Very visual.
 
[Kristin] (1:10:54 - 1:10:55)
How about you, Regina?
 
[Regina] (1:10:55 - 1:11:00)
Here's mine. Natural experiments are a hot flirtation date with causality.
 
[Kristin] (1:11:01 - 1:11:08)
Ooh, I love it. And yours is a little more positive, so I feel like it's balancing out my negativity around causal inference methods.
 
[Regina] (1:11:09 - 1:11:21)
That works. This has been a super fun episode, Kristin. I love how we each got a study to report on and dive into.
 
And very different methods, but same topic.
 
[Kristin] (1:11:22 - 1:11:33)
Yeah, and we covered some pretty meaty stats topics in here, Regina. This has been a lot of fun. Thanks, Regina.
 
[Regina]
Thank you, Kristin, and thanks, everyone, for listening.