Subgroup Analyses: Subpar or Sublime? | NEJM Evidence

This animated video explores some of the potential pitfalls of performing subgroup analyses in randomized controlled trials and explains how to approach potential findings with caution.

[@kbbluesgreens74]

Very helpful; I like these statistical explanation/interpretation videos and the cautions associated with them. Wish there were more

[@lostisle0]

This video raises very good points, but I find it misleading/confusing. If "apple eaters" is the subgroup, it is usually not a treatment of interest, as it seems to be in the "CROP trial" example. Rarely will one run a trial randomizing to "drug/fruit" vs "no drug/fruit", without specifying which drug/fruit to take. Imagine running a trial of antihypertensive vs placebo to treat hypertension, in which patients in the antihypertensive group can take whatever drug they want, from an ACE inhibitor to hydrochlorothiazide. Who would do that? In most subgroup analyses, you have a treatment (say lisinopril), and you want to see if it is better than placebo in a subgroup like "apple eaters." Then the same three issues described in the video arise. First, there is risk of confounding. While confounders such as "daily exercise" may be balanced in the lisinopril vs placebo groups (say 100/1000 in both groups), by sheer chance, it may become unbalanced in a subgroup like "apple eaters." You may have only 1/10 apple eaters in the lisinopril group who exercise daily, but happen to have as many as 5/11 apple eaters in the placebo group who exercise daily. Now placebo may seem better than lisinopril at lowering blood pressure among apple eaters, simply because of enrichment of exercise (which also lowers blood pressure) in the placebo group. Second, there's the issue multiple hypothesis testing. You may test lisinopril vs placebo in apple eaters, pear eaters, orange eaters... separately. Even if we suppose lisinopril is no different from placebo in any these groups, you may find a difference by chance if you use a type I error rate (alpha) of 0.05. Third, power is diminished. As in the example above, if you're comparing 10 apple eaters in the lisinopril group vs 11 apple eaters in the placebo group, you have much less power than comparing 1000 patients in each group in the overall trial.

[@daaj317]

I love that a research journal alerts about the possible "clickbaits" on the papers.

[@embryoman]

really great! thank you!

[@wolfpytlak2786]

thank you for doing these videos