Hot Tubbing: Can soaking in hot water help you run faster?

[Kristin] (0:00 - 0:02)
So maybe this is like endurance hot tubbing? Is this what we're talking about?

[Regina] (0:03 - 0:09)
Oh, I like it. This should be an Olympic sport.

I would excel at this.

[Kristin] (0:15 - 0:25)
Welcome to Normal Curves. This is a podcast for anyone who wants to learn about scientific studies and the statistics behind them. I'm Kristin Sainani.

I'm a professor at Stanford University.

[Regina] (0:25 - 0:31)
And I'm Regina Nuzzo. I'm a professor at Gallaudet University and part-time lecturer at Stanford.

[Kristin] (0:31 - 0:36)
We are not medical doctors. We are PhDs. So nothing in this podcast should be construed as medical advice.

[Regina] (0:36 - 0:44)
Also, this podcast is separate from our day jobs at Stanford and Gallaudet University. Kristin, today we're talking about hot tubs.

[Kristin] (0:45 - 0:48)
Oh, goody. I like hot tubs. Actually, I have a hot tub.

[Regina] (0:48 - 1:02)
I know you do. We use it a lot when I visit. But today we're not just looking at whether hot tubs are fun, because we already know that they are.

We are looking at whether they're a good substitute for exercise.

[Kristin] (1:02 - 1:08)
Substitute for exercise? Wow. So do a hot tub instead of exercising?

That's what you're saying? Basically, yes.

[Regina] (1:08 - 1:27)
Okay, let me give you the background. Apparently, endurance runners like you are always looking for the extra edge in performance. And they do crazy things like go train in the mountains or run around in special suits that make you sweat.

And those things make your body change and get more efficient at running.

[Kristin] (1:27 - 1:54)
I will vouch for the fact that, yes, distance runners are crazy and try anything to get an edge. There's a general ethos of what doesn't kill you makes you stronger, right? And so if something makes you suffer more, then it's going to make you perform better, right?

That's the idea. And I have actually trained in the mountains. After college, I took a year off to run, actually, and I went out to Colorado and trained there for three months at altitude to try to benefit from the altitude training.

[Regina] (1:54 - 2:02)
I want to hear more stories about your crazy altitude training. And I think, Kristin, you're saying no pain, no gain.

[Kristin] (2:02 - 2:04)
No pain, no gain. Yes, exactly.

[Regina] (2:05 - 2:18)
This is the idea. That sort of stuff has been going on for decades. But recently, some people have wondered, hey, do I really need to be suffering quite that much?

Can I just sit in a hot tub and get the same effects?

[Kristin] (2:19 - 2:26)
Yeah, I mean, sometimes you can suffer in a hot tub if you crank it up high enough and force yourself to stay long enough. So maybe this is like endurance hot tubbing. Is this what we're talking about?

[Regina] (2:26 - 2:32)
Oh, I like it. This should be an Olympic sport.

I would excel at this.

[Kristin] (2:33 - 2:52)
People might die, though. I mean, it is actually dangerous if you stay in a hot tub that is too hot for too long. That is why your body is telling you to get out.

But that would be potentially a fun and efficient way if you could get a training boost by just sitting in a hot tub. Less wear and tear on your joints, for sure. Exactly.

[Regina] (2:52 - 3:25)
So today we're talking about one particular study that looked at exactly this. They had elite runners sit in hot tubs for five weeks, 45 minutes at a time. And they wanted to see if they got that training effect, right?

Without any extra training. And I saw that researchers wrote an article on this for a general audience online magazine called The Conversation. Very interesting.

And, Kristin, this was their headline. Marathon training, colon, why hot baths might help you run faster.

[Kristin] (3:25 - 3:27)
Oh, that's a pretty big claim. Isn't it?

Although they said might.

They said might, so.

[Regina] (3:30 - 3:41)
Might. I know, might. That word might is doing a lot of heavy lifting there.

I adapted it and the claim that we will evaluate is getting right to the point. Hot baths help you run faster.

[Kristin] (3:41 - 3:50)
Well, you know, Regina, I have to admit that I'm biased and I really want this to work. That would be great. You just sit in the hot tub and get a training effect.

I love it.

[Regina] (3:50 - 4:01)
I vote that I spend more time sitting in your hot tub instead of doing hills on my bike. And maybe a glass of wine or a little gummy. That's going to count as exercise enhancement, right?

[Kristin] (4:01 - 4:07)
I don't think the wine or the gummy anyone is claiming will improve your running times.

[Regina] (4:08 - 4:14)
Relaxation. OK, I am going to do my own end of one experiment on that regardless. I will get back to you.

[Kristin] (4:14 - 4:15)
Get back to me, yes.

[Regina] (4:16 - 4:23)
And I feel like this is part of our whole series lately of lazy ways to get exercise benefits without actual exercise.

[Kristin] (4:24 - 4:38)
It does kind of fit in with our last episode, which was on exercise snacks, like how little exercise can you do in a day and actually benefit. So, yeah, I think we have a theme here. I feel like we're catering a little.

Like, is this just to get listens, Regina?

[Regina] (4:38 - 4:40)
This is to help me. I like a good lazy life hack.

[Kristin] (4:40 - 4:41)
Lazy life hack. I like it.

[Regina] (4:42 - 5:13)
OK, so that's what we're going to do in this episode. We'll look at the physiology actually first and whether the study actually measured things well, how well the evidence supports that headline. And we are also going to get into a cool statistical technique called best subsets regression.

[Kristin]
Fun.

[Regina]
Before we get started, Kristin, I just wanted to give a little shout out to a listener and a supporter, Zainab, who bought us a coffee. Kristin, do you want to read the great comment that they left for us?

[Kristin] (5:13 - 5:28)
I'm a second year PhD student and have never enjoyed learning as much as I do listening to you guys. I'm currently writing a paper and you guys have had such a positive impact on the process. I am way more critical now in how I communicate my messages.

Thanks a lot. Keep it up.

[Regina] (5:28 - 5:31)
That just made my heart sing. I am way more critical now.

[Kristin] (5:32 - 5:44)
Our job is done, Regina. Our job is done. I love it.

[Regina]
Thank you, Zainab.

[Kristin]
But now, Regina, let's do a little background here. What's the biology here?

How would getting really hot help you potentially?

[Regina] (5:44 - 6:21)
Well, let me back up and actually talk about altitude training first, because the paper said altitude training works one way, heat training works another way, and hot tubs might be somewhere in between. They explained all this, though, Kristin, with a bunch of physiology jargon that I did not understand, so I had to invent a metaphor. The whole cardiovascular system is like a big Amazon delivery system.

The muscles are the customers and you need a lot of super fast delivery of oxygen. So you're always hitting that deliver by tomorrow, please.

[Kristin] (6:21 - 6:25)
Oh, I love that. You need something, it'll show up at your door at 4 a.m., yes.

[Regina] (6:26 - 6:36)
Muscles need oxygen right away. I'm picturing it as, I don't know, like I order a lot of ice cream maybe and need it really fast. My ice cream or cookies.

[Kristin] (6:37 - 6:43)
The problem with ice cream is that I'll order it and then I'll forget that I ordered it and it will sit outside and melt, so that one doesn't work well.

[Regina] (6:45 - 7:09)
Okay, cookies, cookies. Cookies, yes. Okay, so then I broadened out this metaphor, right?

So the lungs are the warehouse where the oxygen gets loaded up for delivery and then the red blood cells, those are the delivery trucks. They travel on the highways and deliver the oxygen to the muscles and the heart is the dispatch logistics center that sends out all the delivery trucks to the highways and the highways are the blood.

[Kristin] (7:10 - 7:11)
Oh, I think that's a great analogy, Regina.

[Regina] (7:12 - 7:20)
So I had to look at what is happening at high altitude. Now, you said that you have done this. Do you know anything about the physiology behind it?

[Kristin] (7:20 - 7:30)
All I know is you get less oxygen and so your body adapts by making more red blood cells to carry more oxygen, something like that. That is the theory.

[Regina] (7:30 - 7:45)
That is exactly what I found. It's that the air is thinner, so there's less oxygen that you get in each breath. You're starving yourself of oxygen, but your body compensates by making more red blood cells.

Those are the delivery trucks.

[Kristin] (7:46 - 7:49)
Yeah, I think that's a really good analogy, actually, Regina. Extra delivery trucks, yeah.

[Regina] (7:49 - 7:59)
And I think the idea is that those extra red blood cells stick around for a while when you come back to sea level, right? So you get a little bit of a boost, like doping afterwards.

[Kristin] (8:00 - 8:19)
Okay, so what happened with you, though? Did it work? I took a year off after college just to run semi-professionally, which meant going off to Boulder, Colorado for three months and taking a few odd jobs, but actually having a lot of time to run in altitude to try to increase my oxygen delivery system.

[Regina] (8:19 - 8:27)
You have had a fascinating life. So you lived in Boulder just to run, just to add these red blood cells.

[Kristin] (8:27 - 8:37)
Yeah, just for three months to do some altitude training. Yes, it's the mecca. Everybody, all the runners go there.

You meet people and you say, oh, I'm a runner. They're like, oh, have you done the Boulder? Boulder.

Everybody runs.

[Regina] (8:39 - 8:49)
Okay, so it seems kind of expensive to keep doing this altitude training, right? Not everyone can afford to just go bum around Colorado for a year.

[Kristin] (8:49 - 9:01)
Well, you know, there's other ways to try to get the high altitude effect. There's cheats that are a little cheaper. So a lot of runners sleep in an altitude tent.

So you put this tent over your bed and it sucks out some of the oxygen.

[Regina] (9:02 - 9:02)
No way.

[Kristin] (9:03 - 9:04)
Yeah, to try to mimic.

[Regina] (9:04 - 9:09)
In your own home, you're starving yourself of oxygen while you sleep.

[Kristin] (9:09 - 9:14)
Yeah, so you get a training effect while you sleep. Runners are very efficient like this. Don't want to waste any time.

[Regina] (9:14 - 9:18)
You guys are crazy, I think is the word you're looking for.

[Kristin] (9:20 - 9:45)
When you want to have that extra edge, like when you're seriously committed to this, you know, that's what you do. You go that extra, extra distance. You have to be a very specific type of creature to want to be an endurance athlete and particularly a distance runner.

Yeah, I'll just make a note, though. I was just a marathon runner. Marathon runners are pretty normal.

It's only the ultra marathon runners. They're crazy and we are normal. So just putting that out there.

[Regina] (9:45 - 9:58)
You know, that's what the 10K runners say about the marathon runners. Point that out. So apparently the latest thing, though, is heat training, doing exercise in heat.

[Kristin] (9:58 - 10:15)
So this is something I haven't heard as much of. Of course, if you are in a very hot climate, you suffer more. So perhaps you're stronger when you then come to a colder climate.

But I haven't really heard much about like using heat as an actual training effect to try to do something physiologically to the body other than suffer more. So tell me about it.

[Regina] (10:15 - 10:37)
Yeah, yeah. I think it's like in the past 10 years they've been doing this. So you still end up with more oxygen delivery tracks, but it's a slightly different method than the altitude thing.

Yeah. So it looks like the heat makes your body expand your blood plasma volume. And that's the watery part of your blood.

[Kristin] (10:37 - 10:38)
Like to cool yourself down, maybe?

[Regina] (10:39 - 11:25)
Yes, exactly. That's what it is. You're bringing more whatever fluid to your skin to cool yourself down.

So in my bad metaphor, it's like the blood is the highway system that you use for the delivery trucks. And we've got Jeff Bezos at Amazon, and he's reacting to the heat by building new lanes on the highway. But at first, the lanes are just empty because you've got the same number of trucks, right?

Plasma is just liquid. But then your body notices you have this extra capacity and it starts building more red blood cell delivery trucks to take advantage of that. And it doesn't happen as fast as when you're starving yourself of oxygen in thin air, but you still end up with more oxygen delivered to your muscles.

[Kristin] (11:26 - 11:28)
Oh, that is actually kind of a cool idea.

[Regina] (11:28 - 11:56)
There is a lot of research that really shows that this is reliable. So I read, Kristin, that some people exercise for like an hour in these special 95-degree heat chambers. This just sounds horrible, right?

But I'm also wondering, with climate change, maybe we don't need these special rooms anymore, right? So just come to D.C. in the 100-degree weather that we just had in July. It was delightful.

It was just like a sauna all the time.

[Kristin] (11:56 - 12:08)
I do remember I did live in D.C. for three months when I was young also, and I used to run to work. And I do remember it being quite sweltering. I'm not sure if it improved my times, but it was hot, yeah.

[Regina] (12:08 - 12:11)
It's just gross. Well, you're suffering, so it must do something. Suffering, yeah.

[Kristin] (12:11 - 12:12)
Right.

[Regina] (12:12 - 13:02)
Okay, but here's the thing. So heat training, now a potential training for improving your performance. But then researchers started to say, hmm, is it the heat itself that's driving the cardiovascular changes?

Maybe you don't need to exercise while you're hot. Maybe you can just be hot.

[Kristin]
Just take out the middle man.

[Regina]
Take out the uncomfortable part of the running. I love it. Okay.

Okay, and that is how researchers started looking at hot tubs. And there's actually a whole bunch of research on hot tubs, saunas, for general health, like does it help you live longer? It's kind of fascinating.

But today we're just focusing on these elite athletes and whether they can shave off a few minutes of their marathon time or whatever.

[Kristin] (13:02 - 13:16)
You know, Regina, those papers and the research on hot tubs and saunas for general health, I do have that on my to-do list for us to sometime cover in Normal Curves because I have to admit, I do really like the sauna and I would love for that to be good for your health. So we'll get there.

[Regina] (13:16 - 13:20)
We will need to do a lot of research for that episode by going to all the saunas in California.

[Kristin] (13:20 - 13:27)
Oh, I think so. A minimum number of sauna trips in order for us to be able to speak with expertise on the sauna.

[Regina] (13:28 - 13:29)
All right.

[Kristin] (13:29 - 13:33)
So I'm dying to hear about this paper though. So tell me, what did they do? What did they find?

[Regina] (13:33 - 13:49)
Right, right. Okay. Published in a journal called Journal of Physiology, November 2025.

So the first author is a PhD student in exercise physiology, Cardiff University. And the senior author is his advisor. And they had 10 participants.

[Kristin] (13:49 - 14:00)
Uh-oh, 10 is full. This gets back to our N of 10 is a dinner party, not a sample size.

[Regina]
An N of 10 is a hot tub party.

[Kristin] (14:01 - 14:14)
Is a hot tub party, yes.

[Regina]
Update it. Okay.

Nine of them were men. They were all mostly in their 20s. They were super fit.

They ran at least five times a week. These were people like you, not like me.

[Kristin] (14:15 - 14:19)
Was there any inclusion criteria where you had to run a certain time or just, you know, people who were pretty fit?

[Regina] (14:20 - 14:24)
I think that there probably was, although they did not say that in the article.

[Kristin] (14:24 - 14:25)
Okay. These seem like real runners.

[Regina] (14:26 - 14:38)
These are real runners. Okay. Now the study design was a case crossover.

And you had five weeks of hot tub intervention, five weeks of washout, and five weeks of the control condition.

[Kristin] (14:38 - 15:23)
We talked about the crossover designs in our last episode, actually on exercise snacks. But just to remind listeners, crossover means that every participant gets all the interventions. So everyone in this study got both the hot tub condition and the control condition.

And we like crossover studies because it's more powerful statistically when everyone serves as their own control. Was the order randomized, Regina?

[Regina]
It was.

It was. Absolutely.

[Kristin]
Okay.

So some people got the control first. Some people got hot tub first. And that was random.

I'll just note what the washout period is. It means that you separate the two interventions. And you do this because if the hot tub actually works, there might be some lingering effects of the hot tub.

And you want to let those wash out before you have the control condition. And five weeks seems like a pretty good washout period.

[Regina] (15:23 - 15:44)
Yep. Yep. I think it was well designed.

It was well thought through on this. Let's talk about the hot tub, the intervention condition. 45 minutes of sitting around in a really hot hot tub five times a week.

And this was on top of usual training and usual diet. And control condition, nothing special. Just normal routine.

[Kristin] (15:44 - 15:48)
That is actually a lot of time in the hot tub. And how hot was really hot?

[Regina] (15:48 - 16:14)
Oh, that is a good question. So they had participants start at 104 degrees Fahrenheit and then increase it gradually as much as they could stand. And the paper said they got up to an average of 107 degrees Fahrenheit with a standard deviation of 2 degrees.

So that means some people got over 107 degrees, which just makes my skin hurt just thinking about it.

[Kristin] (16:14 - 16:22)
That sounds dangerous, actually. It really does. That is pretty hot.

I don't think my hot tub will go up past. It might get up to 105 if I really push it.

[Regina] (16:23 - 16:30)
Well, the researchers kept asking the participants how uncomfortable it was for them. And if it was too comfortable, that's when they jacked up the heat.

[Kristin] (16:31 - 16:57)
So this is really endurance hot tubbing. This is not relaxation time. They are trying to make them suffer.

And it's good that the researchers were there because that means that this was monitored. They were not doing this on their own where they might actually pass out and be quite dangerous. I think, Regina, this episode might need a disclaimer.

We're not suggesting that you try this at home because, actually, it could be dangerous to do 45 minutes of a hot tub at 107 degrees. That's physiologically pushing it.

[Regina] (16:57 - 17:13)
It is. The participants also had to stay in the water up to their neck.

[Kristin]
Oh, no.

[Regina]
But then they started to, I don't know, they couldn't stand it anymore. They were overheating. And the researchers are like, OK, you can periodically stand up, air yourself out a bit, but then sit back down.

[Kristin] (17:13 - 17:13)
Oh, wow.

[Regina] (17:14 - 17:14)
Yeah.

[Kristin] (17:14 - 17:17)
So this is really like a pain tolerance test almost.

[Regina] (17:17 - 17:28)
I know. This is not what I was picturing. I was picturing, like, you know, 1970s hot tub party in Santa Cruz, California, which they didn't have wine or gummies either.

[Kristin] (17:28 - 17:33)
No, this sounds like hardcore hot tub training. Did they get bubbles?

[Regina] (17:33 - 17:36)
Allowed bubbles? Oh, you know, it didn't say.

[Kristin] (17:36 - 17:38)
OK. I want to know if they had bubbles.

[Regina] (17:38 - 17:42)
The bubbles and maybe like the little disco lights that we get.

[Kristin] (17:43 - 17:58)
The lights. I had the lights in my hot tub, yes. Let's get back, though, to that number, sample size 10.

10 is just really small. It is a crossover design, which does increase statistical power. You can get away with less people.

But did they do any kind of sample size calculation to justify 10?

[Regina] (17:58 - 18:13)
Oh, unfortunately, no. So they admitted this, though, at least. They said, well, a couple of previous heat training studies used between 9 and 12 participants.

So we aim for 12 and we got 10. And that was good enough for us.

[Kristin] (18:13 - 18:37)
I'm laughing here. At least they were honest. I mean, I actually I really appreciate the honesty.

Some people would have just made up a sample size calculation to fit that they got 10. Love the honesty. And also, it is a PhD student, so they have to eventually graduate.

So I'm going to give them some amount of pass here, keeping in mind that 10 is very small. And what did they measure for the outcomes? Like, how do we know if the hot tub worked or not?

[Regina] (18:38 - 19:17)
Right. So they measured a bunch of physiological things. The idea is that if hot tubs worked, they'd be able to see all of those heart and blood adaptations over time, right?

All those things in my delivery truck metaphor. And so they counted all of that, the red blood cell, delivery trucks, highway capacity, dispatch center. They had six measures around adaptations in the blood system, 10 measures around cardiac heart adaptations.

And they also measured how uncomfortable people were, how much they were sweating. They measured a lot of things.

[Kristin] (19:17 - 19:26)
That's a lot of measurements. Regina, we talked in this podcast before about multiple testing. So did they designate any of those outcomes as the primary outcomes to help control multiple testing?

[Regina] (19:26 - 20:10)
So they didn't use the word primary or secondary outcomes at all. But they did say, hey, we have two big objectives. And the first was to see how the hot tub condition changed those oxygen delivery system things that I mentioned.

They picked out three, though, in particular. One was the red blood cell delivery trucks. The other one was blood volume, right?

That highway capacity. And the third was heart capacity. How much blood the heart could hold before it would beat each time.

That's in the main pumping chamber. And that's like the dispatch center.

[Kristin] (20:10 - 20:15)
So they didn't call these primary outcomes, but they kind of had these three main outcomes.

[Regina] (20:16 - 20:34)
Three main outcomes, plus a bunch of other stuff. And how do you measure those things? This was a lot of machinery.

It was really cool. They went into a lot of detail, loving detail, about all of their experimental protocol. I am not going to get into that.

People can go geek out on that themselves.

[Kristin] (20:35 - 20:53)
So this was not an easy study to do because there were a lot of complicated measurements that they tried to take. Regina, in all of what you just described, though, we're talking about the Amazon delivery system and your physiology. I didn't hear anything about did they run faster.

So was running faster one of the outcomes of this study or not?

[Regina] (20:53 - 21:08)
Yeah, kind of no. So everything was in the laboratory. So they didn't look at, OK, did your marathon time improve?

But they did look at six measures of fitness or what they called performance.

[Kristin] (21:08 - 21:10)
So I assume, Regina, that maybe VO2 max was in there?

[Regina] (21:11 - 21:18)
Absolutely. There were two different ways to measure VO2 max. And we'll get into some of these when we get to the results because it's kind of interesting.

[Kristin] (21:19 - 21:39)
VO2 max is a typical one that we measure as a measure of fitness. So if that went up, that would be compelling, I think, even if they didn't measure their running times yet. Regina, did they pre-register their study?

We've talked a lot about this pre-registration on this podcast so we could know whether they had pre-planned all of their analyses.

[Regina] (21:39 - 21:42)
Sadly, no. But they did admit it, though. They said we did not pre-register.

[Kristin] (21:43 - 22:00)
Oh, they actually said that in the paper? Wow. So first of all, I think the PhD student should listen to our podcast.

But secondly, I do appreciate the honesty and the awareness of that, hey, our sample size was kind of small. We forgot to pre-register. But at least we're telling you we did those things.

I love it.

[Regina] (22:01 - 22:06)
All right. Let's talk about the results then.

[Kristin] (22:24 - 22:31)
But first, let's take a short break.
Welcome back to Normal Curves. Today, we are asking the question whether taking a good hot tub can improve your running times.

[Regina] (22:31 - 22:39)
Let's talk about the results. Let's start with that oxygen delivery system and then we'll get on to the more interesting ones.

[Kristin] (22:39 - 22:43)
Right. Because those were the main ones we think are kind of like the primary objectives.

[Regina] (22:43 - 23:03)
So each of the three that I mentioned were statistically significant. And that means that they looked at the measurement before and after the five-week hot tub period and before and after the control period. And the improvements were greater in the hot tub period.

[Kristin] (23:04 - 23:20)
So that's great. This Amazon delivery system was significantly improved. But of course, that's not what we really care about, right?

Like I care about that I get a better VO2 max or ultimately that I run faster. So what about VO2 max? That's the result I really want to know, Regina.

Did the VO2 max improve?

[Regina] (23:20 - 23:29)
Right. So they actually measured VO2 max in two different ways. One was the absolute VO2 max.

And I think that is what you're seeing like on your Garmin watch.

[Kristin] (23:30 - 23:39)
Yes. Which mine is getting lower and lower, sadly. But yes.

So I haven't paid attention to it in a while. But yeah, that's a number that might be like 55 or 45, right?

[Regina] (23:39 - 24:01)
Mm-hmm. Okay. So this they measure by putting you on a treadmill and then they keep turning up the speed until basically you're about to fall off and collapse and cry uncle.

And you're wearing a mask at this time. And the mask is able to measure how much oxygen you can take in and use when you're at that maximum intensity.

[Kristin] (24:01 - 24:09)
And more is better. More is better. This VO2 max does correlate very strongly with fitness and very strongly with running performance.

[Regina] (24:09 - 24:39)
Mm-hmm. Sadly, though, Kristin, absolute VO2 max, not significant. There was no significant improvement in hot tubs compared to the control condition.

[Kristin]
Oh, you're bumming me out, Regina.

[Regina]
Okay. But strangely enough, maybe this will make you happier, Kristin.

The relative VO2 max was significantly better in the hot tub condition. And relative VO2 max is just when you take that absolute number and divide it by the person's weight.

[Kristin] (24:40 - 25:07)
Right. We kind of care a little bit less about relative VO2 max because the absolute VO2 max is more correlated to performance. The relative would be like, how fast did you run for your weight?

And it's not usually a thing that we do. We sometimes say like, how fast did you run for your age, but not for weight. So I'm not so impressed that absolute VO2 max didn't come out significant, even if relative did.

I don't know why that would be, too. Did they change weight during the study? Do we know that?

[Regina] (25:07 - 25:19)
They might have. So they didn't report on weight, but I don't know. If you're sitting and boiling in the hot tub for five weeks, maybe you lost a little weight there.

And so when you're dividing out, sometimes these things happen.

[Kristin] (25:19 - 25:45)
Right. So, Regina, why would that be? I mean, if you sit in the hot tub 45 minutes, five times a week for five weeks, maybe you lost some weight.

And if your weight goes down a little because you're like sweating out everything, if the numerator doesn't change, but you divide by something that's smaller, that would make a higher relative VO2 max, right?

[Regina]
Right. Exactly.

[Kristin]
Exactly. So, Regina, how big was that effect, though?

[Regina] (25:45 - 25:52)
It looks like it was about 2.7 milliliters per minute per kilogram. Okay.

[Kristin] (25:52 - 25:59)
I think that's kind of a moderate effect size that people would say was big enough to care about, but that's not a measure we all think in.

[Regina] (25:59 - 26:02)
No, it's not. It looks like it was an improvement of about 4%.

[Kristin] (26:03 - 26:07)
Oh, 4%. I mean, for an elite runner, 4% is actually a lot, potentially.

[Regina] (26:08 - 26:22)
Right, right, right. That is what I thought, although I need to put in this caveat, I didn't get the actual numbers because the researcher did not put the numbers for all this performance stuff in a table. They just put it in figures, bar charts.

[Kristin] (26:22 - 26:27)
Oh, no. They used bar charts for numeric variables?

[Regina] (26:27 - 26:28)
I know.

[Kristin] (26:28 - 26:30)
Which we've said many times in this podcast, don't do that.

[Regina] (26:30 - 26:33)
No, I didn't know whether to be angry or sad.

[Kristin] (26:35 - 26:45)
So the numbers were only contained in the bar charts? They were never given anywhere? Does that mean that you went and used graph to table, though, Regina?

Our boyfriend.

[Regina] (26:45 - 26:56)
That is exactly our boyfriend to the rescue again. They gave a few of the numbers in the text as well, but not as much as you and I would want, not as much detail.

[Kristin] (26:56 - 27:03)
All right. So 4% improvement in relative VO2max. You mentioned there were some other performance variables, though.

Any of the other ones show anything interesting?

[Regina] (27:03 - 27:35)
Yeah. Okay, there was one that was actually significant, and this was what they called treadmill speed at maximal exertion. Okay.

Is that like the speed you can get before you fall off the back? Yes, I think that's exactly what it is. Okay.

Again, don't try this at home. Do not try this at home. This was actually statistically significantly better in the hot tub improvement than in the control, and it translates to about half a mile per hour more.

[Kristin] (27:36 - 27:44)
Half a mile in an hour is actually, I mean, that's not nothing. Right, right. Maybe it's just that the laboratory is really hot and they got used to being in heat or something.

[Regina] (27:46 - 27:50)
So that one was significant, but then there were other ones that were not significant.

[Kristin] (27:50 - 27:57)
Oh, you said there were six total performance variables, so only two out of six were statistically significant, and we definitely have a multiple testing problem here.

[Regina] (27:57 - 28:16)
We do. So running economy, that's how efficiently you can run, that was not significant, and they looked at your heart rate and your speed when you are at what's called the lactate threshold, and this is where you're at moderate intensity, and neither of those were significantly improved in the hot tub condition.

[Kristin] (28:17 - 28:17)
Okay.

[Regina] (28:17 - 28:22)
And they never, like you asked before, never looked at improvement in marathon time.

[Kristin] (28:22 - 28:33)
So performance results, I'm not terribly impressed. But, you know, if all you need to do is sit around in a hot tub for four hours a week, that doesn't sound so bad.

[Regina] (28:33 - 28:45)
I think the problem is finding a hot tub where you are legally allowed to boil yourself alive for 45 minutes, five times a week, and make it affordable. I don't know where you can do that.

[Kristin] (28:46 - 29:01)
Unless you have a hot tub in your backyard. Again, also, I'm worried about the safety of this. I really don't think you should boil yourself alive for 45 minutes, five times a week, that there may be some safety implications.

So don't do this alone. Don't do this at home. And yeah, it's probably expensive.

I guess you can get a gym membership where they have a hot tub or something.

[Regina] (29:02 - 29:06)
Yeah, I don't think they crank them up to like 107 degrees, 109 degrees.

[Kristin] (29:06 - 29:12)
They don't, yeah. So this may not be quite as easy a quick trick as exercise snacks might have been.

[Regina] (29:12 - 29:20)
No, no, sadly. But I'm not going to let that stop me from continuing to go to hot tubs and calling it exercise.

[Kristin] (29:20 - 29:26)
I think you can just go and call it relaxation time. And that's okay, too. We can justify that as well.

Yes.

[Regina] (29:26 - 29:38)
So now, Kristin, I want to talk about their exploratory analysis because they did some interesting things statistically, one good thing and one not so good thing.

[Kristin] (29:38 - 29:41)
They're really stretching out their N of 10 study, aren't they?

[Regina] (29:42 - 29:51)
They are. They had a very specific goal here. They said, okay, VO2max looks like it has improved more with the hot tub condition.

[Kristin] (29:52 - 29:54)
Well, the relative VO2max, though, not absolute.

[Regina] (29:54 - 30:09)
Yeah. Okay. You bring up an interesting and very awkward point because you're right.

It was relative VO2max that was significant. But in this analysis, they used absolute VO2max. They chose the non-significant outcome.

[Kristin] (30:09 - 30:11)
Oh, okay. That's interesting.

[Regina] (30:11 - 30:23)
Don't ask me why.

I have no idea why they did that. Other than they were not expecting absolute VO2max to be non-significant. And they just went with it anyway.

[Kristin] (30:23 - 30:37)
So that would presume that some people did have an improvement because otherwise you can't have a mediator of an improvement if there was no improvement. So some people did, but some people didn't, right? Because on average, absolute VO2max did not improve, but there still was variation in that.

[Regina] (30:37 - 30:53)
Right. Exactly.

Okay. So remember there were about 16 of these measures, 6 of the blood system adaptations and 10 cardiac adaptations. And they said, okay, which ones actually matter?

So you could throw them all into one giant regression.

[Kristin] (30:54 - 31:01)
Wait a minute. We have a sample size of 10. So we cannot have 16 predictors in a regression model of a sample size of 10.

That is a big no-no in statistics.

[Regina] (31:02 - 31:28)
Big no-no. Okay. So they did something better and that was actually legal, right?

They said, okay, let's just find out which subset of these 16 measures best explained VO2max improvements. And we're just going to try all of the subsets. And Kristin, this ends up being a fun combinatorics problem.

And I know you love combinatorics and you teach it. So let's have you do my mini quiz.

[Kristin] (31:29 - 31:34)
Oh, yes. I do love combinatorics and it is one of my favorite things to teach. So go for it.

[Regina] (31:34 - 31:40)
So let's say I have two predictors. How many possible subsets do I need to test?

[Kristin] (31:40 - 31:58)
Okay. So only two predictors. We're starting with a simple case like blood volume and red blood cell count or something.

So with two predictors, if you want to think about all possible subsets, you could have both predictors together. You could have just the first predictor, just the second predictor or neither. So that is a total of four possible subsets.

[Regina] (31:59 - 32:02)
Bingo. 100%. Okay.

What about three predictors now?

[Kristin] (32:03 - 32:44)
Okay. So with three predictors, you can have all three included, all three excluded. You could have three each with just one predictor and three each with two predictors. And if you add that up, you get eight.

And you might start to see a pattern here. So with two, it was two squared is four. And with three predictors, it's two raised to the third power and that is eight.

So if we went up to four predictors, then it would be two to the fourth power, which would be 16. Another way you can think of that is each predictor is either included or not included. So it's binary.

So there's two possibilities for each predictor. And we have to decide included or not included for every single predictor. So we're raising to the power of the number of predictors.

That's how to think of it.

[Regina] (32:44 - 32:52)
Exactly. Okay. Kristin, now we have 16 predictors.

Go. How many?

[Kristin] (32:52 - 33:06)
Oh, that would be two raised to the 16th. And I can't do that in my head very quickly because I'm going to have to go two times two is four times two is eight times two is 16 times two is 32 times two is 64. And I'm going to stop there. 128.

Yeah. 256. Oh, wait.

Okay. I think that's as far as I can get.

[Regina] (33:06 - 33:07)
I can just tell you.

[Kristin] (33:07 - 33:08)
Yeah, just tell me.

[Regina] (33:08 - 33:10)
It is more than 65,000.

[Kristin] (33:11 - 33:20)
Wow. So you're telling me they did regression models with over 65,000 possible models. And I am guessing they did not do this by hand.

[Regina] (33:21 - 33:28)
They definitely 100% had a computer program to do this. There's a package in R, actually, that does it all automatically.

[Kristin] (33:28 - 33:33)
So they used R. They get brownie points for using R and not Excel. Yay.

Yay.

[Regina] (33:34 - 33:47)
Okay. So this is the good thing they did. And okay.

You ready? I'm ready. I'm not going to list all 65,000 models.

[Kristin]
Thank you. Please don't.

[Regina]
The winner, though, had only two predictors.

[Kristin] (33:47 - 33:55)
Oh, well, that's great. So this algorithm landed on a nice, simple, and parsimonious model. Excellent.

And what were those two predictors?

[Regina] (33:55 - 34:01)
Number of red blood cells and blood volume capacity in the heart's pumping chamber. Okay.

[Kristin] (34:01 - 34:02)
That sounds reasonable.

[Regina] (34:03 - 34:07)
But Kristin, there is a big problem with this that I did not notice at first.

[Kristin] (34:08 - 34:09)
Oh, that doesn't sound good.

[Regina] (34:10 - 34:52)
This is not good. Right. I glossed over it.

I'm going to highlight it now. So remember I said they wanted to figure out which of these biological mechanisms best explained the hot tub therapy VO2 max improvement over the control therapy, right? So that means my outcome is change.

So it's like I have the before and the after of the five weeks of the hot tub therapy, and I have the VO2 max before and after of the five weeks of the control treatment. And so I want to look at the improvement in hot tub. So I subtract after minus before and do the same thing for control.

And then I subtract those. So I get the relative improvement.

[Kristin] (34:52 - 34:54)
We call that a difference in difference. Two differences.

[Regina] (34:57 - 35:16)
Okay. So that's what we should have done. My outcome is the change.

But what the researchers actually put in the model was just VO2 max as the outcome, not the change. And what they did is try to statistically adjust for that before and after. And hot tub versus control condition.

[Kristin] (35:17 - 35:50)
So let me just see if I can picture the model here, Regina. Our outcome is the VO2 max at a given time point, not the improvement. And you're saying that they also put into the model, in addition to these biological predictors, pre or post and control versus hot tub, which means what they're actually doing is answering a very specific question, which is what is the effect of the biological predictors on VO2 max?

If we ignore, if we control for or hold constant, whether the person is in the control or hot tub condition and whether it's before or after the therapy.

[Regina] (35:51 - 36:19)
That is absolutely right. So it's like, okay, say you had one runner and he started out with like amazing VO2 max. Of course, we're going to expect that he's going to have more red blood cells and a bigger heart because they already know the physiology of this.

So the model is going to point to that and say, oh, look, that's what's important. But they're not looking at whether hot tubs actually changed anything in him at all. Instead, they're saying, what's behind VO2 max?

Ignore everything else.

[Kristin] (36:19 - 36:50)
So, I mean, that is not a bad question to ask. What best correlates with your VO2 max, your blood volume, your red blood cells or whatever? That is a reasonable question to ask, but it's just not the question that we think they were trying to answer in this study.

Regina, do you think they just messed up or do you think that they were actually trying to say, hey, the hot tub improved these biological predictors and we're going to connect these biological predictors to VO2 max? Were they trying to make an A to B and B to C and then therefore A to C leap? Or do you think they just messed up the model?

[Regina] (36:50 - 37:06)
I think they messed up the model because they claimed throughout the abstract and the conclusions and the results that they found these biological predictors or underpinnings of the improvement in hot tub effect.

[Kristin] (37:06 - 37:35)
This is why it's so important to really think through what are you putting into your model? What does that model mean? And what question is it answering?

And I think people often overlook this when they're throwing together regression models and they're not careful enough.

[Regina]
Yep. Yep. Sadly.

[Kristin]
They definitely needed a statistician on this study. PhD students can't always afford to have a statistician on their study.

No, maybe they can listen to our podcast though. They need to listen to our podcast. It's the cheap version of a statistician on your study.

[Regina] (37:36 - 37:52)
The poor man's version.
But this actually gets to another point, Kristin. I'm glad you brought this up because the journal made the peer review record completely available in the supplementary materials. So I could go in and see what the actual suggestions were from the peer reviewers.

[Kristin] (37:52 - 38:13)
Oh, I love this. I love this when you can go read the peer reviews. This is, by the way, a very modern approach that some journals are doing now.

And it allows you to see, did the peer reviewers pick up on any of these things that we're talking about? You know, it's really good to see the whole peer review process because sometimes it's actually a little alarming when you see the peer reviews because they're not as robust as you might hope. So what did you find here?

[Regina] (38:13 - 38:25)
I was alarmed.

[Kristin]
Oh, no.

[Regina]
This was not good.

I was actually kind of shocked at this. So first of all, Kristin, there was only one peer reviewer for this paper.

[Kristin] (38:26 - 38:30)
Really? That is actually really unusual. Usually there are two.

So I'm surprised at that.

[Regina] (38:31 - 38:45)
And that peer review, I would call at best a cursory review, not any substantial point. For example, the peer reviewer asked that the authors use the abbreviation SV instead of writing out stroke volume.

[Kristin] (38:46 - 39:03)
Oh, my goodness. I give the opposite review comment. I would have told them don't use SV, write out stroke volume, because I, as a reader, don't want to see SV because I'm going to think of like SVU criminal edition, whatever that show is, rather than I'm not going to remember that it means stroke volume.

So I'm the opposite.

[Regina] (39:04 - 39:23)
The peer reviewer only made like six points. It was so skimpy. And two of them were about this like abbreviations or formatting.

So I know. They also did things like ask the authors to speculate about whether saunas would be the same as hot tubs and what was going on with the mitochondria.

[Kristin] (39:24 - 39:32)
Oh, that's all speculating past the data. Again, I actually would have the opposite review comment. I think I need to peer review this peer reviewer, Regina.

I think so.

[Regina] (39:33 - 39:36)
And OK, it gets even worse, though.

[Kristin] (39:36 - 39:36)
No way.

[Regina] (39:36 - 40:09)
Because the peer reviewer actually introduced an error, like a statistical interpretation error. Yeah. So remember in that last subset regression we were talking about, the best model was red blood cell count and this heart capacity, right?

And the peer reviewer told the authors to report that it was red blood cell count that was the strongest substantial predictor of VO2max and that the heart capacity measure simply added additional explanatory power.

[Kristin] (40:09 - 40:12)
But how would you conclude that from this best subsets regression?

[Regina] (40:12 - 40:36)
You cannot. There is nothing in there. In fact, that's the whole point of what the researchers did was go in and do all of the possible subsets so they could find what was the most important package, the most important predictors.

You can't peel them apart. But the researchers, they just added that line to their manuscript exactly as they were requested to do.

[Kristin] (40:37 - 41:39)
They just want to make the reviewer happy and get it done with. That happens with peer review sometimes. A reviewer suggests language and when you're trying to get a paper accepted, it's very tempting to just make that change without fully thinking through whether it's actually supported.

I mean, with peer review, I think it's always good to think about the suggestions that the peer reviewers make, but you don't have to make all of them if you don't agree. Sometimes, yeah, it's just easier to give the reviewer what they want, but you can hold your ground on something that's not right.

[Regina]
Yeah, exactly.

[Kristin]
Okay, well, Regina, that was just the exploratory analysis. Even if they messed it up a bit, it wasn't the main claim of the paper. And I think we're ready to rate our claim for the episode today, which I think you based on that headline, but took away the might.

So the claim was that hot baths help you run faster. And how do we evaluate the strength of evidence behind claims on this podcast with our 1-to-5 smoosh rating scale, where 1 means little to no evidence for the claim and 5 means a lot of evidence for the claim. And what do you think on this one, Regina?

[Regina] (41:39 - 42:15)
Okay, you're going to be shocked at this, Kristin. I am going to give this three smooches.

[Kristin]
Oh, wow.

[Regina]
I know. Okay, here's my justification. No, I've not really seen any evidence of hot tubs helping you run faster.

Maybe, you know, cardiovascular fitness, maybe more red blood cells, but the whole relationship isn't clear. I mean, I get why they put the running faster thing in the headline. You know, it's kind of clickbait.

I don't think it's going to help normal people like me run faster. So that's kind of misleading. Okay, but here's the thing.

I was accused of smooch inflation for the exercise snacks.

[Kristin] (42:16 - 42:19)
Oh, the last episode where you gave it three.

[Regina] (42:19 - 42:37)
Yes. I gave it three and I was accused of smooch inflation, which is true because I wanted that exercise snack hack to be true. So I feel like I need to do the same thing here. I don't really believe it.

I really want lazy exercise hacks to be true. So I'm going to go ahead and give this three smooches.

[Kristin] (42:37 - 43:44)
I appreciate the consistency in your smooch inflation and the honesty about your biases. Yes.

[Regina]
What about you?

[Kristin]
So on this one, I'm going to go 1.5 smooches. I'm giving it a little more than the exercise snacks only because I really want this one to be true as well because, you know, sitting in the hot tub sounds like a fun thing to do to get free fitness from. I do believe that there's some biological plausibility to this, but we're not looking at the body of research here.

We're just looking at this one paper. I don't think that this one paper has a ton of strong evidence in it. Maybe some of these biological measures were improved.

The VO2max, since it was only relative and not absolute and they looked at a lot of things, I'm not totally convinced by that. But I'm going to go 1.5 smooches, a little better than the bottom here because at least one thing I did like is they were very honest in this paper about like, hey, yes, our sample size is too small and hey, we didn't pre-register. At least they were upfront.

I also think this paper suffered by not getting a better peer review and they used R. So I'm going 1.5 smooches and still hoping this to be true. And I think we should still go to saunas next time you're here.

[Regina] (43:44 - 43:47)
Absolutely. I'm behind that. Okay, methodological morals, what do you have?

[Kristin] (43:47 - 44:00)
Regina, I'm going to go with this model where they didn't really seem to understand what the model was actually answering because this comes up very often. So mine is a model can be very good at answering the wrong question.

[Regina] (44:01 - 44:09)
I love that so much. This is behind, I think, a lot of the studies that we encounter on this podcast, isn't it?

[Kristin] (44:09 - 44:15)
It is. And especially when the models get a little complicated, that's where things can really go awry. How about you, Regina?

[Regina] (44:15 - 44:22)
I'm going to focus on that crazy peer review. Yeah. Okay.

The label peer reviewed does not guarantee carefully reviewed.

[Kristin] (44:22 - 44:30)
I think that's a good takeaway. Everybody should realize that, yes, peer review is better than no peer review, but sometimes it doesn't really add much.

[Regina] (44:30 - 44:38)
Right. I think it's important to know if you're the public or the journalist, peer review is not magic.

[Kristin] (44:38 - 44:46)
Yeah. It's not always perfect. Great. Well, this has been fascinating, Regina.

I enjoyed this a lot.

[Regina]
More hot tabs, more saunas. Absolutely.

Thanks, Kristin.

[Kristin] (44:46 - 44:48)
Thanks, Regina. Thanks, everyone, for listening.