Post
Conversation
![David Watson 🥑](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="96" height="96"><rect fill-opacity="0"/></svg>)
And I wanted to talk a bit more baout whether Poldermans really killed 800,000 people. Here's what I wrote in the newsletter:
The 800,000 number comes from cardiologists Graham Cole and Darrel Francis, who are also two of the authors of the meta-analysis that found a 27% increase in deaths if beta-blockers are given before surgery.
The number just comes from identifying how many surgeries occurred subject to the guidelines over the course of five years in Europe, and then multiplying out the higher fatalities implied by the 27% increase in deaths.
So is it a good estimate? If I had to place a bet I'd bet it's too high. A rule of thumb I use: effect sizes are overestimated wildly more often than they're underestimated. If you're looking at an effect size estimate, the real effect size is most likely a lot smaller.
This isn't always the result of any specific malfeasance or carelessness. Ime it's usually a some combination of publication bias and the experiment being conducted with more rigor and care and adherence to the studied standard than will be present in real life circumstances.
But whether it's medical research or development research I think effects are usually smaller than reported, and I'd predict that's so in this case. Thus my equivocal phrasing in the newsletter - I think the deaths here were high but likely lower than the straightforward math.
But people who hate the 800k number are in my experience incredibly loath to say "too high by a factor of 2" or similar in favor of declaring the whole conversation "unscientific".
Cole and Francis wrote an online article in the European Heart Journal that included the estimate, which was published and then yanked on the grounds that the article 'contains scientific statements' [and therefore should require peer-review].
The authors were annoyed: they observed in a response that the 'scientific statement' in question is literally just "used basic arithmetic on 3 values in the public domain." In this dispute I am basically entirely sympathetic to the authors.
I don't think there's any world in which it's scientific to publish the effect size estimate but unscientific to publish an estimate of the actual public policy implications of that effect size estimate.
I do think that the effect size estimate is largely higher than the actual effect size, and that adherence to the guidelines was probably somewhat lower than the estimate assumes. You can debate how much to adjust for that.
But I feel very strongly that it is not unscientific (one person even claimed that giving any estimate of deaths at all would be "misinformation"!) to discuss the public policy implications of science. I think the reluctance to talk numbers ends up minimizing the stakes of fraud.
So while I would be very excited to see an analysis arguing that the death toll was more like 400,000 or more like 100,000 - or that the real effect size is null - I commend Cole and Francis for actually doing the math out loud, and challenge their critics to do the same.
One other thing: the newsletter got wrong the type of surgeries that Polderman's DECREASE trial series studied: they were non-cardiac surgeries. I corrected this this morning, and I'm sorry for the error.
A rich history of #innovation and a mission-focused culture. Find out how 12-time Inc.5000 honoree ’s mission, to empower members to live a life without limits, fuels its continued growth. #medicalalert #ad
Hi Kelsey, I took a look at the meta-analysis. The 27% increase has p=0.04. Additionally, they found that beta-blockers HELP with non-fatal side effect (myocardial infarction).
Moreover, though it's a "meta-analysis", it is primarily driven by a single study, the POISE trial.
There are plenty of ways to qualify that number further, none of which are within reasonable expectation for a popular discussion.
For instance, you could go to the original CES analysis and pull the CI for the ‘27%’ and get a range. Or you could compare different surgical…