Cancer Blood Tests: Are they ready for primetime? Part 1

[Regina] (0:00 - 0:07)
So the signal for cancer is going to be weaker, right? Fewer needles, more haystack.

[Kristin] (0:07 - 0:27)
Exactly. Cancer turns out to be a much harder signal detection problem than prenatal testing.

Welcome to Normal Curves.

This is a podcast for anyone who wants to learn about scientific studies and the statistics behind them. I'm Kristin Sainani. I'm a professor at Stanford University.

[Regina] (0:27 - 0:33)
And I'm Regina Nuzzo. I'm a professor at Gallaudet University and part-time lecturer at Stanford.

[Kristin] (0:33 - 0:38)
We are not medical doctors. We are PhDs. So nothing in this podcast should be construed as medical advice.

[Regina] (0:38 - 0:43)
Also, this podcast is separate from our day jobs at Stanford and Gallaudet University.

[Kristin] (0:44 - 0:53)
Today, we're going to talk about liquid biopsies. And this is the idea that you could take a single tube of blood and use it to detect cancer anywhere in the body.

[Regina] (0:53 - 1:03)
Liquid biopsy. Kristin, are we doing science fiction now? Because that's what that sounds like.

I just give you some blood and a single test tells me if I have cancer.

[Kristin] (1:04 - 1:15)
It does sound like science fiction. But we actually have a blood test on the market right now that claims to do exactly this to detect 50 cancers with one blood test.

[Regina] (1:15 - 1:22)
I confess, I really had no idea. I'm honestly not up to speed on this at all. But it sounds exciting.

[Kristin] (1:22 - 1:45)
It is exciting. And maybe outside of Silicon Valley, people aren't following this as closely as we are here in Silicon Valley, perhaps. But Regina, I do believe that this is the future of cancer detection.

I actually think that we are going to have a blood test someday that detects most cancers. I believe that somebody is eventually going to get this right.

[Regina] (1:46 - 1:50)
Kristin, listen to you. You are sounding uncharacteristically optimistic.

[Kristin] (1:52 - 2:12)
But the biology here is plausible. The question for me, though, is whether the technology as it exists today is ready for primetime. Is this going to be ready tomorrow or in 50 years?

That's, that's the uncertainty I'm leaving in. And we're actually going to tackle this question over the course of two Normal Curves episodes.

[Regina] (2:12 - 2:16)
That's right. Two episodes because there is so much to talk about.

[Kristin] (2:16 - 2:51)
Exactly. And the goal of these tests, of course, is to catch cancer earlier than we currently catch it. Because catching cancer earlier can increase your chances of survival.

So these tests are called multi-cancer early detection tests, or MCED tests. And Regina, there are many companies going after this. But we're going to focus on one company and one test, because this test is arguably the furthest along in the multi-cancer early detection race.

And the company is called GRAIL.

[Regina] (2:51 - 2:58)
GRAIL. Oh, that's a clever name, because they are searching for the holy grail of cancer testing, screening. Yeah, you've got it.

[Kristin] (2:58 - 3:03)
And the test is called GALERRI, like Gallery of Cancers, I suppose.

[Regina] (3:05 - 3:11)
That doesn't sound beautiful. I'll just say. No, no.

Galleri with an I though, right?

[Kristin] (3:11 - 3:43)
Yes, spelled with an I at the end, not a Y. And Galleri is currently commercially available. You can actually buy it.

This is the one that's been in the news a lot lately as well. It has been because the company just released results from a huge randomized clinical trial of the test performed in England. It's called the NHS-Galleri trial.

And NHS is the National Health Service of England. They partnered with the company GRAIL to run this trial.

[Regina] (3:43 - 3:50)
So a randomized clinical trial, not just observational studies. This is good. That's how they're going to get some stronger evidence then.

[Kristin] (3:51 - 4:27)
Yeah, the fact that they did a randomized, blinded, controlled trial is very exciting, actually. And, Regina, the active part of the trial is complete. Now, there is no peer-reviewed, published paper on the trial yet.

So I'm going to give the caveat up front, we don't have all the data yet. Too bad. The company did release a press release earlier this year.

They were a little light on details. And then they just released some actual data at a big cancer conference in Chicago last month. And these limited results did generate quite a few headlines.

[Regina] (4:28 - 4:31)
I bet they did. That is catnip for reporters.

[Kristin] (4:31 - 5:18)
Absolutely. So here are some examples of those headlines, Regina. Blood test helps reduce late stage cancer diagnosis.

Screening with a multi-cancer blood test reduced the most advanced cancers. A single blood test finds more early cancers than Britain's screening programs combined.

[Regina]
Well, how about that?

That sounds all very, very positive.

[Kristin]
Ah, yeah, but wait a second, Regina, because I'm not done yet. Here are three more headlines about the same study and same results.

Grail's cancer detection test fails in major study. Holy Grail's blood test for dozens of cancers said to be rejected by NHS after failing early diagnosis trial. This once hot cancer detection company's stock got cut in half after a failed trial.

[Regina] (5:19 - 5:29)
Okay, now we're in the exact opposite. Now it sounds, what, stupendously negative. How could all of these be from the same study?

Were half the reporters drunk or something?

[Kristin] (5:30 - 6:32)
I think they might have been, yes. Regina, I think this is why this is a perfect study to discuss here on normal curves, because the results are not straightforward. And that's what we do here.

We look at nuance. So, Regina, I decided to base the claim for this two-part episode off of the headlines. And since I'm kind of a pessimist when it comes to evidence, I went with the negative framing.

So here is the claim for these episodes. GRAIL's Galleri multi-cancer detection test is not ready for prime time.

[Regina]
Ooh, I like it.

[Kristin]
And before we get started, a quick shout out to David Rind for suggesting this episode. He is a physician and the chief medical officer at the Institute for Clinical and Economic Review. And he's also a patron.

Yeah, David has given us a few episode ideas now, hasn't he? He has. And we want everyone to know that we love to hear from listeners.

And we sometimes run with your ideas. So reach out with your questions and ideas. normalcurvespodcast at gmail.com.

[Regina] (6:32 - 6:41)
Okay, Kristin, you mentioned that we are breaking this into two episodes. So give us a quick preview of what we're covering today versus next time.

[Kristin] (6:41 - 7:07)
Sure. Today, we're going to cover all the background, the biology behind these tests, the history of Grail and the Galleri test, and the results from earlier non-randomized studies of the Galleri test. Basically, Regina, everything other than the randomized trial.

Then in part two, we'll dive into the randomized trial, how it was designed and how to reconcile the two very different sets of headlines that the trial's results generated.

[Regina] (7:08 - 7:18)
I am also going to tease that second episode by saying there will be some statistical sleuthing and some interesting surprises in the data. Goodies, goodies.

[Kristin] (7:18 - 7:28)
Yes, that's what we do here on Normal Curve. So good teaser, Regina. Okay, Regina, let's start by talking about the biology, about how these kinds of tests actually work.

[Regina] (7:29 - 7:37)
Right, because if you tell me a blood test can find cancer anywhere in my body, my first question is going to be, hmm, really? How?

[Kristin] (7:38 - 7:53)
Yeah, the basic idea is really cool. Cells in our bodies are constantly dying and releasing tiny fragments of DNA into our bloodstream. Scientists call these fragments cell-free DNA.

[Regina] (7:53 - 8:02)
Cell-free DNA. So DNA that has broken free of their cells like their jail cells. I'm picturing them on the run, on the lam.

[Kristin] (8:03 - 8:17)
Oh, I like that analogy. Well, it's kind of like in the movies if like the entire jail exploded into a bunch of pieces. And the convicts have escaped.

Yes. Because the cells have died and broken apart and they have shed remnants of their DNA into the body.

[Regina] (8:18 - 8:36)
Oh, okay. That is quite an exciting movie.

And now these remnants on the on the lam on the run are floating around in my blood.

Lots of little scraps of DNA from cells all over my body. And they're just having a giant pool party in there.

[Kristin] (8:37 - 8:57)
The convicts are now in a giant pool. We like to mix metaphors and analogies here. Yeah, yeah.

I like it. Most of those scraps actually, though, of course, Regina come from perfectly normal, healthy cells. But if a cancer is present, some of those DNA fragments may come from tumor cells.

[Regina] (8:57 - 9:07)
And there are probably more fragments from healthy cells than there are from tumor cells. So it's like a needle in the haystack, swimming pool, convict roundup kind of thing.

[Kristin] (9:08 - 9:25)
Yeah, I think if we really want to do this analogy, it's like the whole town blew up. And along with that, the jail cell blew up. Some of the convicts escaped into the blood.

But they're not the majority of people who escaped the explosion. Yes.

[Regina] (9:25 - 9:28)
Right, right. Needle in the haystack still. Right.

Yes.

[Kristin] (9:28 - 10:14)
And Regina, actually, this is not a new concept. We actually use this idea in a totally different area of medicine, in prenatal screening for chromosomal abnormalities, things like Down syndrome. So before the early 2010s, the screening tests for chromosomal abnormalities involved an ultrasound and a blood test.

And these tests were not very accurate. They produced a lot of false positives, requiring a lot of women to get invasive tests like amniocentesis, you know, when they stick you with that long needle. Just to give you a sense for healthy 30 year old women, something like 95% of positive screening tests turned out to be false positives.

In fact, that happened to me with my daughter, I got a false positive on this screening test.

[Regina] (10:15 - 10:37)
Oh, that must have been stressful. Because false positives are not harmless. In cases like this, right?

Getting a false positive is going to make people worry, understandably. And then they're going to have to invest time and money and follow up tests. And I think, Kristin, people often underestimate when they're thinking about screening tests, the cost of false positives.

[Kristin] (10:37 - 10:57)
Yeah, my false positive was stressful, right? I had to have an invasive test. And there was a week or two in there of me reading every paper in the literature on the subject, you know, counting dots off the scatter plots, to convince myself that there was a high probability that mine was a false positive.

Everything turned out fine, but still not fun.

[Regina] (10:57 - 11:07)
No, not fun. And this is true with any screening test. Not just this prenatal screening test.

They all have costs associated with false positives.

[Kristin] (11:07 - 12:09)
Yeah, true for cancer screening, just as much as for prenatal screening. Absolutely. But Regina, in the early 2010s, unfortunately, after I was done having kids, prenatal screening underwent a revolution and became dramatically more accurate.

And just to give you an idea, for healthy 30-year old women, remember, we said about 95% of those positive tests were actually false positives? Well, that dropped down to something more like 50%.

[Regina]
That is a dramatic improvement.

What changed?

[Kristin]
So remember, we're looking for problems in the baby's DNA. But the old tests were not directly looking at the baby's DNA, they were looking at indirect clues, things like a subtle change on the ultrasound.

Back then, the only way to actually get a hold of the baby's DNA was to stick a needle into the woman's uterus. The revolution came when scientists realized that there is a non-invasive way to get the baby's DNA. They could detect cell-free DNA from the baby and from the placenta, actually, circulating in the mother's bloodstream.

[Regina] (12:10 - 12:16)
Oh, and that brings us back to today, because these are the same kinds of DNA fragments that the cancer tests are looking for.

[Kristin] (12:17 - 12:28)
Exactly. In both cases, you're looking for DNA that's genetically different from the person whose blood you're testing. In pregnancy, it's the fetus and the placenta.

In cancer, it's the tumor.

[Regina] (12:28 - 12:40)
Okay, all of that makes sense. But you said fetal testing, highly accurate, they've got that down pat now, it sounds like. But the cancer testing, I take it is harder, not quite yet, down pat.

[Kristin] (12:41 - 12:55)
Yes, the cancer testing is much harder. So for one thing, the fetus and the placenta are bigger than most tumors. So they are shedding way more DNA into the mother's bloodstream.

Cancers, especially early cancers, they shed way less DNA.

[Regina] (12:56 - 13:03)
So the signal for cancer is going to be weaker, right? Fewer needles, more haystack. Exactly.

[Kristin] (13:04 - 13:08)
Cancer turns out to be a much harder signal detection problem than prenatal testing.

[Regina] (13:08 - 13:26)
And Kristin, it makes sense to me now that they can detect DNA from the fetus because the fetus's DNA is different from the mother's DNA, but the cancer cells come from your own body. So it's the same DNA. What are they looking for in the tumor DNA, then?

How do they spot it?

[Kristin] (13:26 - 13:37)
That is a great question. Regina, the most obvious difference that we can look for is mutations in the DNA, right? Cancer cells often accumulate spelling mistakes in their DNA that normal cells don't have.

[Regina] (13:37 - 13:41)
Oh, so you're just looking in the blood for DNA with those mistakes and mutations.

[Kristin] (13:42 - 13:49)
Yes. But mutations aren't the only signal. Cancer doesn't just cause mutations.

It can alter DNA in other detectable ways.

[Regina] (13:50 - 13:54)
Detectable, like genetic fingerprints, let's say.

[Kristin] (13:54 - 14:07)
That's a great way to think about it. And different companies are chasing different cancer fingerprints in the DNA. Grail's fingerprint of choice is something called DNA methylation.

And Regina, do you remember what that is?

[Regina] (14:07 - 14:12)
I absolutely do not remember. Remind me what DNA methylation is.

[Kristin] (14:12 - 14:34)
It's actually pretty cool. Sometimes a small chemical tag called a methyl group gets attached to the DNA. And these tags can act like on-off switches, telling the cell to dial down or even turn off certain genes.

I like to think of it as a little sticky note on the DNA telling the cell which genes should be active and which should stay quiet.

[Regina] (14:34 - 14:41)
Oh, I like the sticky note analogy. Okay, so cancer cells have different sticky notes than normal healthy cells.

[Kristin] (14:41 - 15:06)
Exactly. Yes, cancer cells often develop abnormal methylation patterns. So Grail's Galleri test sequences millions of DNA fragments from your blood and looks for methylation signatures associated with cancer.

And if it thinks cancer is present, it also tries to predict the type of cancer. Because it turns out, for example, that lung cancer has different methylation patterns than, say, breast cancer.

[Regina] (15:07 - 15:18)
Oh, so that's cool. It's not just asking, is there cancer here, but also what type of cancer? So Kristin, tell us more about this Galleri test.

How was this one first developed?

[Kristin] (15:19 - 16:01)
Regina, the history here is actually really interesting, because part of the origin story of Grail actually starts with prenatal testing. So around 2014, scientists at the company Illumina were doing these cell-free DNA blood tests for pregnant women, screening for chromosomal abnormalities. But occasionally, they saw some strange patterns.

They saw multiple chromosome gains and losses that didn't make sense for a fetus. And one of the scientists who noticed these odd results was Dr. Meredith Halks-Miller. She and her colleagues realized that the DNA pattern looked less like fetal DNA and more like tumor DNA.

And when doctors followed up on some of these women, several of them turned out to have previously unsuspected cancers.

[Regina] (16:02 - 16:03)
Wow, that's pretty amazing.

[Kristin] (16:03 - 16:18)
And this is what sparked the idea that we might be able to detect cancer DNA floating in the blood. And that led to the creation of GRAIL, founded in 2016. And Regina, as you can imagine, it attracted a lot of investor money.

[Regina] (16:18 - 16:21)
Yeah, the whole thing sounds like a potential gold mine.

[Kristin] (16:21 - 16:42)
Yeah, cancer screening applies to everyone. So we've got lots of potential customers and repeat customers too, because you would presumably have this like every year. So to develop the test, GRAIL analyzed blood samples from thousands of people with and without cancer, and they used machine learning to identify the methylation patterns associated with cancer.

[Regina] (16:42 - 17:12)
Kristin, we haven't really talked about machine learning yet on the podcast. So how about I just give like a short version of this. Basically, machine learning is pattern recognition on steroids.

You give it thousands of blood samples from people with cancer and thousands from people without cancer. And then the machine learning algorithm goes in and find all of the patterns that would best distinguish one group from the other.

[Kristin] (17:13 - 17:22)
Right. And not only does it distinguish cancer from non-cancer, it also tries to find the patterns that best distinguish one type of cancer from a different type of cancer.

[Regina] (17:22 - 17:27)
Machine learning, kind of amazing, if they can actually get it to work here.

[Kristin] (17:28 - 17:47)
Yeah, someday we'll have a whole episode, I think, on machine learning. But that's a great summary for today, Regina. The first step to showing whether the test works or not is actually to test it on an entirely new set of people, not the same people that they use to develop the test.

And this is called validation when we go out and test it on a new set of people.

[Regina] (17:47 - 18:08)
Right, validation. Validation is important because you want to make sure the machine learning algorithm hasn't just identified some chance, weird, random patterns that happen to be in your study sample, but are not present anywhere else in the population, right? You don't want it to just focus on the noise or the flukes.

[Kristin] (18:08 - 18:36)
Absolutely. And, you know, Regina, we talked about validation back in the sugar sag episode on that cadaver study that was so interesting. So for anyone who wants more details on validation, I'll send them there.

GRAIL conducted a validation study, which they published in the Annals of Oncology in 2021. And that study looked at about 2,800 people with cancer and about 1,200 people without cancer. And they focused on estimating the tests sensitivity and specificity.

[Regina] (18:37 - 18:45)
Sensitivity and specificity. We talked about these statistics in our episode on diagnostic testing, which is a great review, actually, if anyone needs it.

[Kristin] (18:46 - 19:26)
That episode actually sets up this episode really well, because today we're going to be referring to a lot of diagnostic test statistics. So just a caveat, Regina, the Galleri algorithm has undergone some refinements since 2021. So there may have been slight improvements in the sensitivity and specificity since then.

But I'm going to focus on this study because this is the only place where I could find sensitivity broken down by cancer stage. And that's pretty important. So let's start with sensitivity.

And it did vary by stage. It was 17% for stage one cancers, 40% for stage two, 77% for stage three, and 90% for stage four.

[Regina] (19:27 - 20:03)
Wow, they did vary a lot by stage, didn't they? So sensitivity, first a review sensitivity is what percent of true cancer cases the test managed to actually pick up. So stage one tumors in the study Galleri picked up only 17% of them.

But then on the other hand, stage four tumors Galleri picked up 90% of them. And 90% is really high. But Christian, when I think about it, it makes sense what you said before stage four would be easier to detect because stage four cancers shed more DNA.

[Kristin] (20:04 - 20:17)
Exactly. So Galleri is clearly missing some of the earlier cancers, you know, the ones we'd really like to catch. But picking up 17% of stage one cancers earlier is definitely better than nothing.

It could definitely save lives.

[Regina] (20:17 - 20:40)
Also, the specificity of the test was high at 99.5%. Ah, specificity, meaning that 99.5% of people without cancer correctly tested negative, which means that there were only 0.5% false positives. That is really good, because as we talked about false positives are bad.

[Kristin] (20:40 - 20:50)
Yes. And finally, this paper found that the test was able to identify the origin of the cancer, like was it breast or lung, correctly about 90% of the time.

[Regina] (20:51 - 20:52)
90%. That is amazing.

[Kristin] (20:52 - 21:02)
It is. But Regina, this study was what we call a case control study, which means the researchers already knew who had cancer and who didn't.

[Regina] (21:02 - 21:21)
Case control study. Yes. This means they could not calculate the thing that we really care about, which is if I test positive, what are the chances I really do have cancer.

And the diagnostic testing episode, we explained in detail why case control studies do not let you calculate this number at all.

[Kristin] (21:22 - 22:09)
And this number is called the positive predictive value, as we talked about in that episode. Regina, kudos to the GRAIL researchers. Unlike some of the researchers we talked about back in that diagnostic testing episode who did this wrong, the GRAIL researchers did not try to calculate positive predictive value from a case control study.

They knew better than that. So good job. Good for them.

Another problem with case control studies is it really doesn't do us any good if we are able to identify people who have cancer after we already know that they have cancer, right? In a case control study, cases have already been diagnosed with cancer. We of course need to show that the test picks up cancers before people get a cancer diagnosis through a different means.

So in other words, we need to study people who don't yet have a cancer diagnosis and see if the test picks up cancers in any of them.

[Regina] (22:10 - 22:18)
Right, which is called a prospective screening study. And it is that kind of study that will give us all these good answers that we need.

[Kristin] (22:18 - 22:48)
Exactly. So GRAIL did do two major prospective screening studies, Pathfinder 1, which was started in 2019, and Pathfinder 2 started in 2021. These were both non-randomized studies without control groups, and I'm going to get back to some details on those in a minute.

But a lot happened with GRAIL in 2021. So in 2021, GRAIL launched that NHS Galleri randomized trial. That's the one that generated all the recent headlines.

[Regina] (22:49 - 22:53)
And the one we're going to dig into in detail with statistical sleuthing in part two.

[Kristin] (22:53 - 23:00)
Yes. Regina, also in 2021, GRAIL released the Galleri test commercially in the United States.

[Regina] (23:00 - 23:10)
That is interesting. You mean it was on the market before they had solid evidence from a randomized trial that it actually did anything useful?

[Kristin] (23:11 - 23:44)
Yes, yes. It was not FDA approved, and it's not covered by insurance. But your doctor can still prescribe it if you want to pay out of pocket for it.

And Regina, full disclosure, I've actually had the Galleri test. So my doctor offered it to me because of my family history of cancer. Now he was very upfront, he did tell me very clearly that no one knows if the test works, and that the evidence was still being collected.

But I decided to spend the roughly $950 anyway, because I'm a worrier, and I thought a negative result might be reassuring, give me some peace of mind.

[Regina] (23:44 - 23:51)
I can see how a negative result would be worth it. It could have some real psychological value, especially, like you said, for worriers.

[Kristin] (23:53 - 24:00)
Ironically, though, I had a negative Galleri test about nine months before I was diagnosed with cancer.

[Regina] (24:00 - 24:14)
Oh, that is beyond ironic. Because you had peace of mind. Yeah, peace of mind that was incorrect.

So you are a false negative here, despite that high specificity of the test that we just talked about. Ouch.

[Kristin] (24:14 - 24:24)
Yeah, I had stage one breast cancer, which Galleri is not good at detecting. So it's not surprising that the test missed my cancer. But it does mean that I was technically a false negative.

[Regina] (24:24 - 24:29)
So Kristin, might you have any personal biases here? You want to declare anything?

[Kristin] (24:30 - 24:48)
So I might. But I think I went into this with a pretty open mind, because GRAIL has always been upfront that the sensitivity for stage one breast cancer is low. So my false negative doesn't really sway me one way or the other.

And I'm pretty good at looking at the evidence and sticking to the evidence.

[Regina] (24:48 - 24:51)
You are. You are open minded. I'll give you that.

So I believe you.

[Kristin] (24:51 - 24:59)
Okay, Regina. So I now want to talk about those two Pathfinder studies, the non-randomized perspective screening studies that I mentioned.

[Regina] (24:59 - 25:33)
Sounds good. But first, a break. Welcome back to Normal Curves.

We're talking about the GRAIL Galleri multi-cancer early detection test and results from the two Pathfinder studies.

[Kristin] (25:34 - 25:42)
Yes. Again, these were non-randomized, no-control group perspective screening studies where everyone got the Galleri test.

[Regina] (25:42 - 25:49)
Right. And the randomized trial was the controversial one. So I'm going to now tease the part two episode when it comes out.

Will be fun.

[Kristin] (25:50 - 26:04)
Great teaser, Regina. Yes, we're going to be talking about the randomized trial in the upcoming episode. Regina, I feel like also in this episode, I don't see an easy way to get sex into this episode, unfortunately.

So I'm just giving up on that one. This is too serious of a topic, I think.

[Regina] (26:04 - 26:08)
It is. I will do my best. I will do my best.

[Kristin] (26:08 - 26:29)
Okay, I'll put that out as a challenge for you. If you can't get it into part one, we'll work on it for part two. Okay.

[Regina]
All right. Deal.

[Kristin]
All right.

So let's start with Pathfinder 1. Again, that began in 2019, and they enrolled about 6,600 adults over 50 who did not have signs or symptoms of cancer. And if they had had cancer in the past, they had to be more than three years done with treatment.

[Regina] (26:29 - 26:46)
So some people in the study were actual cancer survivors then. A lot, actually. It was nearly a quarter.

That's a lot. So this is a relatively high-risk population, right? Because if they've already had cancer once, they're at higher risk for getting it again.

[Kristin] (26:46 - 27:30)
Yeah, exactly. But they may have been more interested in a study like this because they were worried about cancer since they'd already had cancer. But we need to keep this in mind.

This population may not be totally representative of the general population. Right. All right.

In Pathfinder, everyone received a Galleri test at the start of the study at baseline, and then they were followed for 12 months, and they did not get any other special workups for cancer. They were just told to continue with any routine cancer screening that they happened to be due for. Of course, participants who had a positive Galleri test underwent additional testing to look for cancer.

And Pathfinder 2, which was started in 2021, used a very similar design. It was simply much larger. They enrolled more than 35,000 adults over 50.

[Regina] (27:30 - 27:39)
Oh, that is a lot of people. Okay, so we are talking about two year-long studies. And are we ready to talk about the results?

What did they find?

[Kristin] (27:39 - 27:52)
So in Pathfinder 1, there were 92 positive Galleri tests, and 35 of those turned out to be cancer. So the positive predictive value was 35 divided by 92, which is 38%.

[Regina] (27:52 - 28:16)
Okay, just a reminder, positive predictive value is the thing that we really care about a lot, which is if I have a positive test result, what is the chance that I actually have cancer? And Kristin, I want to just give some context on here. 38% might seem low, but positive predictive value in a screening test, that's actually pretty good.

It is, yes. Yeah.

[Kristin] (28:16 - 28:34)
And out of the approximately 6,500 people who tested negative on the Galleri test, there were 86 cancers found in them by other means, which means the negative predictive value was 98.6%.

[Regina] (28:34 - 29:06)
Okay, it's my role to be review girl today. Review Regina. Negative predictive value means if you test negative, what's the probability you truly are cancer free? 98.6% is pretty high. But Kristin, now that I'm thinking about it, they probably did not catch every false negative, right? Because you said they were not actively looking for cancer beyond that Galleri test. And some false negative cancer cases might have been missed in that one year follow up then.

[Kristin] (29:06 - 29:51)
Yeah, almost certainly that some of the false negatives were missed. People had cancer that wasn't caught during that 12 month period. So the negative predictive value is definitely an overestimate.

But no matter what, it's still pretty high. Because even if we had twice as many false negatives, the negative predictive value is still going to be quite high.

[Regina]
Right, right.

Okay, that was Pathfinder 1. What about Pathfinder 2?

[Kristin]
So with Pathfinder 2, the numbers were slightly better, but still in the same ballpark.

Positive predictive value was 60%. And negative predictive value 99.2%. And it's possible that that improvement had to do with some refinements in the algorithm, or it could just be, you know, differences in the population tested or the increased precision of Pathfinder 2. All of those can be true.

[Regina] (29:51 - 29:58)
Right, but still all in the same ballpark, both of them. Consistent results. I would call those very consistent.

Yeah.

[Kristin] (29:58 - 30:08)
And Regina, that's all I'm going to discuss on Pathfinder 2, because we don't have much more on that study, because up to date, they have only presented the data from that study at conferences.

[Regina] (30:10 - 30:26)
Conferences, right, which are often not peer reviewed. They're often very fast, very skimpy on details. And this makes it easy for researchers to present results in, let's say, a more favorable light than they deserve.

Yeah. To bring some skepticism.

[Kristin] (30:27 - 31:12)
To be a little cynical here, they're presenting the results in a way that they help might please maybe investors more than reviewers. Right, right. And much less material for us to fact check in those short presentations.

And GRAIL hasn't made its data like Pathfinder 2 data readily available to outside researchers. Maybe they've shared a lot more data with the FDA with regulators, but certainly not with the public. One side note, I am, Regina, resisting a major temptation here, just because we have way too much else to cover.

I'm going to resist explaining why their favorite statistic from Pathfinder 2, where they say something like, their tests caught 6.5 fold more cancers than routine screening. I am going to resist picking that one apart and explaining why it's completely misleading.

[Regina] (31:12 - 31:23)
I know how difficult it was for you to resist that temptation. Okay, congratulations, but we are going to put it in show notes. So you can still explain it just in show notes.

[Kristin] (31:24 - 32:06)
I am exhibiting a lot of restraint here.

But yes, it will definitely go in the show notes. Yes. Yeah.

Regina, unlike Pathfinder 2, we do have published results from a peer-reviewed paper from Pathfinder 1. Hooray! And so I want to talk some more about Pathfinder 1, because the results from this study were published in The Lancet in 2023.

[Regina]
Ah, Lancet is a good journal. So what did they find?

[Kristin]
One thing that I really like about The Lancet paper is Figure 4, because it shows all 35 cancers that were detected by Galleri and that study, including each cancer type and stage.

And these data are super interesting. So first of all, 17 of the 35 cancers detected, almost half, were blood cancers rather than solid tumors.

[Regina] (32:06 - 32:16)
Oh, that is interesting. It kind of makes sense, right? If you're measuring DNA in the blood, then blood cancers might be easier to find.

Yeah.

[Kristin] (32:17 - 32:24)
Totally. Actually, only about 10% of cancers in the U.S. are blood cancers. So Galleri is clearly picking up blood cancers disproportionately.

[Regina] (32:24 - 32:30)
This is good to keep in mind. So it might be better picking up some kinds of cancers than others. For sure.

[Kristin] (32:30 - 32:35)
And Regina, the significance of detecting these blood cancers is actually a little unclear.

[Regina] (32:35 - 32:43)
But Kristin, picking up blood cancers, it sounds like it would be a good thing, that it would be helpful. So why is the benefit unclear here?

[Kristin] (32:44 - 33:06)
Yeah, well, for example, 6 of the 17 blood cancers were stage 1 or 2 follicular lymphoma. And follicular lymphoma is interesting because it's often very slow growing, it has excellent long-term survival, and sometimes it isn't treated at all. Sometimes doctors simply monitor it.

So the clinical significance of finding it earlier, not entirely clear.

[Regina] (33:06 - 33:42)
This does make sense. So if we have a group of people with follicular lymphoma, it sometimes doesn't matter whether we pick up their cancer earlier or later, because it may not change either the treatment or the outcome very much. And Kristin, I think this is a hard thing to understand about cancer screening, because it seems paradoxical, right?

But catching a person's cancer earlier doesn't always mean that anything in the person's life is necessarily improved. Very paradoxical.

[Kristin] (33:42 - 34:12)
It is. It's really counterintuitive. But it turns out that earlier detection doesn't automatically translate into better outcomes. And in fact, in Pathfinder 1, among all of those blood cancers detected by the test, there were only a handful for which we could make a strong case that earlier detection was likely to improve outcomes.

And we see a similar picture if we look at the 18 solid tumors, the non-blood cancers. For example, five of those 18 solid tumors were recurrent metastatic breast cancer, stage 4.

[Regina] (34:13 - 34:24)
Stage 4, metastatic, breast cancer. These were women who'd had breast cancer in the past, at least three years before the study, but now that breast cancer has come back and spread in the body.

[Kristin] (34:24 - 34:57)
Yes. And again, it's not really clear that detecting this earlier helps them clinically. It's already stage 4, it's not curable, and it may have been found through symptoms pretty soon anyway.

In fact, when I went through the list of solid tumors, I estimated that maybe four to eight of those tumors were cancers where we could make a reasonably strong case that earlier detection may have improved outcomes. And the clearest examples I found were a stage 1 bile duct cancer, a stage 1 small intestine cancer, and a stage 2 pancreatic cancer.

[Regina] (34:57 - 35:00)
Those are very specific. So why those, Kristin?

[Kristin] (35:01 - 35:37)
Well, those are ones that we don't have screening for and are often detected later. And there's some evidence that detecting them earlier may help survival. So for example, only about 30% of pancreatic cancers are typically diagnosed at stage 1 or 2.

So the fact that Galleri caught a pancreatic cancer at stage 2, that's very interesting. It's very promising. Now, the five-year survival rate for stage 2 pancreatic cancer is still not great.

It's only around 15 to 30%. But that's dramatically better than for stage 4 disease, where five-year survival is only about 1 to 3%.

[Regina] (35:38 - 36:06)
This makes more sense now, Kristin. So if I'm understanding you correctly, you're saying pancreatic cancer is often deadly, but it's also often caught quite late. So catching it at stage 1 or stage 2 could really help some patients with survival.

And not perfect, as you said, but this is where a cancer detection test like Galleri could arguably, on average, benefit the population.

[Kristin] (36:06 - 36:34)
Right. But, Regina, I want to point out that our listeners might be noticing just how many caveats and hedge words we are using here. We are usually much more, you know, direct with our language.

But we are using a lot of hedge words because this is all just speculation and guesswork. We really don't know who benefited, if anyone, from the Galleri test. And this is the problem with the Pathfinder studies, right?

There is no control group. So we don't have a way to estimate what would have happened without the test.

[Regina] (36:35 - 37:15)
Right. So this is one of the most confusing things, I think, about these kinds of screening studies, because we're dealing with counterfactuals, things that did not actually happen, that were not facts. What would have happened to these people if they had not gotten the test, even though they did get the test?

There's no way for us to look at this alternative timeline universe where they didn't get the test. We can't peek over there. This is not science fiction, sadly.

So, as you said, the best we can do is to have a randomized control group. That's the closest we have to being able to observe this secret counterfactual universe.

[Kristin] (37:16 - 37:21)
Exactly. Regina, I feel like we're giving people flashbacks to math here when we're referring to counterfactuals.

[Regina] (37:22 - 37:53)
Ooh, ooh, Kristin, this might be a good place to bring in some sex.

[Kristin]
Ooh, go for it.

[Regina]
Okay.

All right. Bear with me on this. Patience.

Counterfactual things that did not happen, and you're trying to picture what would have happened if it did happen if you had swiped right instead of left. You would have ended up, who knows, because you did not swipe right, but you might have ended up with the love of your life and living happily ever after. But no, you swiped left, and that's it.

You're going to die alone.

[Kristin] (37:53 - 39:12)
Oh, I like the analogy. This is how the dating apps get you, right? It's always that counterfactual in the background.

Your soulmate. You might have just missed him, right?

[Regina]
Swiped right on everything. All right.

[Kristin]
Yes, Regina, this is why we need randomized controlled trials, because it's the only way to try to get at what would have happened without the test. And I just want to spend a little time, though, talking about why detecting a cancer earlier might not actually benefit the patient. Because as we've said, I think that's very counterintuitive.

So let's talk about some of the reasons why. So some of the cancers caught by Galleri, they might have been detected pretty soon anyway, without the test, you know, maybe through clinical signs or routine screening, we don't know. And we don't know if those extra few weeks or months that Galleri bought them, those may have made no difference in the patient's outcome, right?

Also, for some aggressive cancers, catching them a little earlier may not change the course of the disease. So for example, those metastatic breast cancers, it's kind of already too late. So maybe there's nothing that can be done.

And then another thing is for some slow growing cancers, the cancer might never have caused problems during the person's lifetime. This often happens with prostate cancer, for example, men die with it rather than of it. And when a screening test finds those cancers, it's called over diagnosis, we've detected more cancers, but we haven't actually helped the patient.

[Regina] (39:13 - 39:40)
I think that was actually a great rundown, Kristin, of explaining this very counterintuitive idea. And this applies to all of cancer screening, right, not just this, this Galleri. And it gets at why we cannot simply count the number of cancers that a test detects, right, and use that to tell us how beneficial the test is for the population.

[Kristin] (39:40 - 40:22)
It's not the simple numbers counting. Right.

So we need a control group to tell us what would have happened if people didn't get the test. We just talked about that. But we also need a study where we look at endpoints that matter to the patient to getting the cancer detected, it may or may not matter to the patient, we need endpoints that matter to the patient.

The best one, the one that everybody would really like to look at is fewer cancer deaths. But there may also be other important things like people got less treatment, they had a better quality of life. Or maybe if we detect fewer advanced cancers, because advanced cancers often lead to death, we need studies that look at those endpoints, not just the number of cancers detected.

And that's why the randomized controlled trial that we're going to talk about in part two is so important.

[Regina] (40:23 - 40:32)
All of this is pointing to why GRAIL's randomized controlled trial is so important, why everyone was really focused on the results of this.

[Kristin] (40:32 - 40:48)
Regina, I want to give GRAIL a lot of credit, because they actually pulled off a randomized controlled trial, which is kind of amazing. But Regina, for the first time ever, we're going to, on Normal Curves, end an episode on a cliffhanger.

[Regina] (40:48 - 40:52)
Oh, cliffhanger. I like it. Who shot JR? That's the cliffhanger.

[Kristin] (40:53 - 40:55)
I never watched that, but I do remember it. Yes.

[Regina] (40:55 - 40:57)
I am. I am old. Who shot JR?

[Kristin] (40:58 - 41:18)
The cliffhanger here would be something more like, did they find statistical significance? I'm not sure if that gets as many people back for episode two.

[Regina]
That's not quite as exciting, no.

[Kristin]
Did the stock crash? Maybe that's, maybe that gets more people back. Yeah.

[Regina] (41:19 - 41:20)
Right, right.

[Kristin] (41:20 - 41:37)
All right, so we are going to wait until part two of this two-part series to discuss the NHS Galleri trial and its results. And remember, those are the results that generated headlines ranging from, blood test helps reduce late stage cancer diagnosis, to, GRAIL's cancer detection test fails in major study.

[Regina] (41:39 - 41:47)
The study that is both like groundbreaking, novel, new idea and breathtaking disaster, all rolled up in one. Exactly.

[Kristin] (41:47 - 41:51)
Same study, same data, completely opposite, contradictory headlines.

[Regina] (41:52 - 41:57)
I want episode part two to help me with my cognitive dissonance, really.

[Kristin] (41:58 - 42:29)
You got it. In part two, we'll try to reconcile those conflicting headlines, we'll separate the hype from the evidence, and we are going to dig deeper into the data than most of the other press coverage out there. So if you really want the nuance and the specifics, that's what we do on Normal Curves. That's exactly what we do.

I think all those investors out there who are wondering whether to sell or buy the stock, I mean, we could get a big audience out of that. We do better at looking at the data and predicting what's going to happen with this company than anybody out there, I think.

[Regina] (42:30 - 42:32)
Don't we need to put a disclaimer if we say that?

[Kristin] (42:35 - 42:43)
Well, we're not going to tell anybody to invest or not, but we're going to give them more, more depth into the data, and they can take and run with it how they see.

[Regina] (42:43 - 42:48)
Right, right. We're better than everyone else, but don't listen to us.

[Kristin] (42:48 - 43:04)
We can't predict the future. But I do think there's things that we see in the data that other people don't. And you know, I think we can make a lot of money off of this, Regina, getting investors to listen to our podcast.

Silicon Valley, I'm in the heart of it, right? I think there's something in that, right?

[Regina] (43:04 - 43:08)
Oh, interesting. Okay, let's keep this in mind. Part two, coming up.

[Kristin] (43:08 - 43:10)
All right. Thanks, Regina, and I'll see you next time.