Statistical significance vs. clinical significance

• Core Concept:
  • Statistical Significance: Evaluates if findings are due to chance ($p<0.05$, $95\%$ CI excludes null: $0$ for diff, $1$ for ratio); driven entirely by sample size ($n$).
  • Clinical Significance: Evaluates if the effect size actually matters to patient care (morbidity, mortality, QoL); defined by Minimal Clinically Important Difference (MCID); independent of $n$.
• Sample Size ($n$) Impact:
  • Very large $n$ (Overpowered): Trivial differences (e.g., SBP $\downarrow 0.5\text{ mmHg}$, $p<0.001$) reach statistical significance without any clinical utility.
  • Very small $n$ (Underpowered): Massive clinical differences (e.g., mortality $\downarrow 20\%$, $p=0.08$) fail to reach statistical significance due to Type II error ($\beta$).
• High-Yield NBME Exam Rules:
  • Calculate ARR/NNT: Never rely solely on $p$-value or relative risk reduction (RRR); a $50\%$ RRR with an ARR of $0.01\%$ is clinically insignificant.
  • Surrogate Endpoints Trap: Statistical improvement in lab markers (e.g., $\downarrow$ LDL, $\downarrow$ HbA1c, $\downarrow$ tumor markers) does not equal clinical significance unless hard clinical outcomes ($\downarrow$ MI, $\downarrow$ death) improve.
  • Evaluating CIs:
    • Statistical significance: CI does not include null ($0$ for difference, $1$ for RR/OR/HR).
    • Clinical significance: Point estimate and lower CI bound exceed the MCID threshold.
  • Negative Trials: If $p>0.05$ in a small trial, the correct answer is usually “Study was underpowered to detect a difference,” not “The drug is ineffective.”