Evidence level: B
联合分层不是免费的——每多切一刀,格子里的样本就少一块。格子太小,判别 AUC 的”提升”只是过拟合的幻觉;格子够大,联合分层才真正优于族裔单层。
正文
上一篇文章把”分层粒度”问题落到了一个可执行方案:比较不分层、仅族裔、年龄×性别×族裔联合分层三种方案的三质判别 AUC。但它留下了一个更致命的工程问题没回答——联合分层的每个格子,到底需要多少人? 这个问题不解决,”联合分层更好”就只是小样本上的假象。
先算分层的账。联合分层是乘法关系:3 个族裔 × 2 个性别 × 5 个年龄档 = 30 个格子。参考区间领域有一条硬性下限——CLSI EP28-A3C 指南规定,用非参数法建立参考区间每组至少需要 120 人(PMID 36266618、PMID 33380887)。按这个下限,30 格就是 3600 人。而 2022 年一项基于 3697 名健康人的研究更狠:非参数法建立 TSH 参考区间需要 850 人,转换参数法只要 239 人(PMID 36266618)——同一指标,方法不同,样本量需求差三倍多。这意味着”每格 120 人”根本不是普适安全线。
格子样本量为什么直接决定判别力?因为参考限的估计精度按 1/√n 衰减。抽样误差公式 ME% = 100/√N 给出直观刻度(PMID 33380887):n=30 时误差约 18%,n=120 时约 9%,n=200 时约 7%。格子越小,三质特异阈值的置信带越宽,阈值本身就越像噪声。
更麻烦的是判别 AUC 的乐观偏倚。小格子里,三质分类模型的每变量事件数(EPV)极低——30 人/格意味着每种体质只剩十来人。而 C 统计量在小样本上的过拟合是系统性的:Steyerberg 的模拟显示,表观 ROC 面积 0.74 会被期望低估 0.07,独立验证实测低估 0.09(PMID 12812818);另一项研究确认,EPV 低于 10 时乐观偏倚显著(PMID 24966219),而”EPV≥10 就安全”这条规则本身缺乏依据(PMID 27881078),即使 EPV 超过 10 也可能出问题(PMID 21411281)。2025 年一项基于 20 万住院患者的模拟进一步指出:非参数模型下,bootstrap 乐观校正会系统性高估 AUC,且增大样本量也压不下去,只有 10 折交叉验证在所有场景下稳定(PMID 41391734)。
把两边合起来,就得到一条权衡曲面。横轴是格子样本量(30/50/100/200),纵轴是三质判别 AUC,两条曲线:族裔单层(每层人多、估计稳,但年龄与性别的残余差异没校准)与联合分层(校准更细,但格子小、乐观偏倚大)。可检验的预测是:在低样本量区,联合分层的 AUC 会虚高,一旦做外部验证就跌回族裔单层之下;在高样本量区,联合分层的真实 AUC 反超。两条曲线之间必然存在一个交叉拐点——这个拐点就是”联合分层到底要多少人”的答案,而不是拍脑袋的 120。
还有一层叠加限制:HRV 测量自身的信度。同一批 HRV 参数的样本量需求跨度极大,从 19 人到 300 人不等,取决于选哪个参数(PMID 37176559);6 分钟记录的重测信度 ICC 才刚过 0.7(PMID 15286929),而 SDNN 在短期记录里最不稳定(PMID 16362542)。格子里测的本身就是带噪信号,等于给已经紧张的样本量雪上加霜。
要把这条曲面画出来,实验设计得满足几个条件。第一,必须是多中心大样本,否则”格子样本量”根本扫不起来——单中心凑不出三十个族裔×性别×年龄格。第二,三质分型要用同一套金标准,且金标准自身的信度必须先量化,否则分层的收益会被金标准噪声稀释掉。第三,格子样本量按 30/50/100/200 四档做重抽样,每档重复多次得到 AUC 的分布而非单点估计——只有这样才分得清”虚高”与”真实提升”。第四,判别 AUC 必须同时报表观值与交叉验证值,两条线的差值就是乐观偏倚的直接度量。
解法有两条。第一,用多层模型代替硬切层:部分池化(partial pooling)让样本少的格子自动向总体均值收缩,样本多的格子保留自己的估计,在偏差与方差之间取最优(PMID 38365599)。第二,用子群可信度准则卡关:判断一个分层是否可信,要看是否为预先指定、方向是否预判、交互是否显著等(PMID 24449319);而现实中 207 项 RCT 里只有 6% 正确预设了方向,多数子群声明可信度很低(PMID 22422832)。三质联合分层同理——不能因为分层后 AUC 涨了就当它真实。
所以联合分层的正确用法是:先用格子样本量梯度(30/50/100/200)做重抽样建模,拟合”置信带宽度-AUC”曲线定位拐点,再用交叉验证确认提升不是过拟合,最后用多层模型兜住不达标的格子。达不到拐点的联合分层,不是更精细,只是更乐观。
证据与局限
已验证: 参考区间样本量需求随方法与指标变化巨大(PMID 36266618),非参数法每组 120 人是下限而非普适线(PMID 33380887);小样本下 C 统计量存在系统性乐观偏倚,EPV 规则不可靠(PMID 12812818、PMID 24966219、PMID 27881078、PMID 21411281);非参数模型的 bootstrap 校正会高估 AUC 且不随样本量收敛(PMID 41391734);HRV 参数样本量需求 19-300 人、重测信度受限(PMID 37176559、PMID 15286929、PMID 16362542);部分池化可自动收缩小格子估计(PMID 38365599);子群效应可信度需专门准则把关(PMID 24449319、PMID 22422832)。
部分验证: 分层粒度的收益与代价在参考区间领域已有成熟框架(Harris-Boyd 准则、CLSI 指南,PMID 15223704),但移植到三质判别 AUC 场景只是方法学类比。
待验证: “格子样本量-AUC 权衡曲面”的交叉拐点在三质场景中的真实位置;低样本量区联合分层 AUC 虚高幅度是否超过其校准收益;多层部分池化能否同时保住小格子的校准收益与判别稳定性。
相关阅读
证据等级: B
参考文献: PMID 36266618, PMID 33380887, PMID 12812818, PMID 24966219, PMID 27881078, PMID 21411281, PMID 41391734, PMID 37176559, PMID 15286929, PMID 16362542, PMID 38365599, PMID 24449319, PMID 22422832, PMID 15223704
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Trade-off Surface of \”Grid Sample Size-AUC\” in Three-Stage Validation: How Many People Are Needed for Joint Stratification?
Evidence level: B
Joint stratification is not free—every additional cut reduces the sample size within each grid. If the grid is too small, the apparent “gain” in discrimination AUC is merely an illusion of overfitting; if the grid is sufficiently large, joint stratification truly outperforms single-stratification by ethnicity alone.
Main Text
The previous article addressed the “granularity of stratification” by proposing an executable solution: comparing three-dimensional discrimination AUC across three schemes—no stratification, ethnicity-only stratification, and joint stratification by age × gender × ethnicity. However, it left a more critical engineering question unanswered: How many people are needed in each grid for joint stratification? Without resolving this, the claim that “joint stratification is better” remains a spurious finding based on small samples.
Let’s calculate the cost of stratification. Joint stratification follows a multiplicative relationship: 3 ethnicities × 2 genders × 5 age bands = 30 grids. The Reference Interval domain has a hard lower limit: the CLSI EP28-A3C guideline stipulates that at least 120 individuals per group are required to establish a reference interval using non-parametric methods (PMID 36266618, PMID 33380887). Adhering to this lower limit implies a requirement of 3,600 people for 30 grids. However, a 2022 study based on 3,697 healthy individuals was even more stringent: establishing a TSH reference interval via non-parametric methods required 850 individuals, whereas the transformation parameter method only needed 239 (PMID 36266618)—the sample size requirement differs by more than three times for the same metric depending on the method. This demonstrates that “120 per grid” is not a universal safety threshold.
Why does grid sample size directly determine discrimination power? Because the precision of reference limit estimation decays at a rate of $1/\sqrt{n}$. The sampling error formula $\text{ME\%} = 100/\sqrt{N}$ provides an intuitive scale (PMID 33380887): error is approximately 18% at $n=30$, ~9% at $n=120$, and ~7% at $n=200$. Smaller grids result in wider confidence bands for the three-dimensional specificity threshold, causing the threshold itself to resemble noise.
More problematic is the optimism bias in discrimination AUC. In small grids, the number of events per variable (EPV) for three-dimensional classification models is extremely low—30 people per grid means only a handful of individuals for each constitution type. Overfitting in the C statistic under small samples is systematic: Steyerberg’s simulations show that an apparent ROC area of 0.74 is underestimated by 0.07 in expectation, while independent validation confirms an underestimation of 0.09 (PMID 12812818); another study confirms that optimism bias is significant when EPV is below 10 (PMID 24966219), yet the rule stating “EPV ≥ 10 is safe” lacks empirical basis (PMID 27881078), as problems can still arise even when EPV exceeds 10 (PMID 21411281). A 2025 simulation based on 200,000 hospitalized patients further indicated that under non-parametric models, bootstrap optimism correction systematically overestimates AUC and does not converge even with increased sample size; only 10-fold cross-validation remains stable across all scenarios (PMID 41391734).
Combining both sides yields a trade-off surface. The x-axis represents grid sample size (30/50/100/200), and the y-axis represents three-dimensional discrimination AUC. Two curves emerge: ethnicity-only stratification (larger samples per layer, stable estimation, but residual differences by age and gender are not fully calibrated) versus joint stratification (finer calibration, but smaller grids and larger optimism bias). The testable prediction is: In low-sample regions, the AUC for joint stratification will be inflated; once external validation is performed, it will drop below ethnicity-only stratification. In high-sample regions, the true AUC for joint stratification surpasses it. Between these two curves must exist an inflection point—this is the answer to “how many people are needed for joint stratification,” not an arbitrary figure like 120.
There is also a superimposed constraint: the reliability of HRV measurements themselves. The sample size requirements for HRV parameters vary drastically, ranging from 19 to 300 individuals depending on the selected parameter (PMID 37176559); the retest reliability ICC for 6-minute recordings is just barely above 0.7 (PMID 15286929), while SDNN is the least stable in short-term recordings (PMID 16362542). The signal measured within the grid is inherently noisy, compounding an already strained sample size.
To plot this surface, the experimental design must satisfy several conditions. First, it must be multi-center with large sample sizes; otherwise, “grid sample size” cannot be adequately sampled—no single center can generate 30 grids of ethnicity × gender × age. Second, three-dimensional phenotyping must use the same gold standard, and the reliability of the gold standard itself must be quantified beforehand; otherwise, the benefits of stratification will be diluted by noise in the gold standard. Third, grid sample sizes should be sampled via resampling at four levels (30/50/100/200), with multiple repetitions at each level to obtain the distribution of AUC rather than a single-point estimate—only then can one distinguish between “inflated values” and “true improvement.” Fourth, discrimination AUC must report both apparent values and cross-validated values; the difference between these two lines is a direct measure of optimism bias.
There are two solutions. First, use multi-level models instead of hard stratification: partial pooling allows grids with few samples to automatically shrink toward the overall mean, while grids with many samples retain their own estimates, optimizing between bias and variance (PMID 38365599). Second, use subgroup credibility criteria as a gatekeeper: determine whether a stratification is credible by checking if it was pre-specified, whether the direction was anticipated, and whether interactions are significant (PMID 24449319); in reality, only 6% of the 207 RCTs correctly pre-specified the direction, and most subgroup declarations have low credibility (PMID 22422832). Joint stratification for three dimensions follows the same logic—just because AUC increases after stratification does not make it true.
Therefore, the correct usage of joint stratification is: first, build a resampling model using grid sample size gradients (30/50/100/200) to fit the “confidence band width-AUC” curve and locate the inflection point; second, confirm with cross-validation that the improvement is not overfitting; finally, use multi-level models to safeguard grids that do not meet the threshold. Joint stratification below the inflection point is not more refined; it is merely more optimistic.
Evidence and Limitations
Verified: Reference interval sample size requirements vary drastically by method and metric (PMID 36266618); 120 per group for non-parametric methods is a lower limit, not a universal line (PMID 33380887); under small samples, the C statistic exhibits systematic optimism bias and EPV rules are unreliable (PMID 12812818, PMID 24966219, PMID 27881078, PMID 21411281); bootstrap correction for non-parametric models overestimates AUC and does not converge with increased sample size (PMID 41391734); HRV parameters require 19–300 individuals with limited retest reliability (PMID 37176559, PMID 15286929, PMID 16362542); partial pooling can automatically shrink estimates for small grids (PMID 38365599); subgroup effect credibility requires specific criteria for oversight (PMID 24449319, PMID 22422832).
Partially Verified: The benefits and costs of stratification granularity already have a mature framework in the reference interval domain (Harris-Boyd criteria, CLSI guidelines, PMID 15223704), but transplanting this to the three-dimensional discrimination AUC scenario is merely a methodological analogy.
To be Verified: The true location of the inflection point on the “grid sample size-AUC trade-off surface” in the three-dimensional context; whether the inflation of joint stratification AUC in low-sample regions exceeds its calibration gains; whether multi-level partial pooling can simultaneously preserve calibration gains for small grids and discrimination stability.
Related Reading
- Calibration Stratification for Three-Dimensional Phenotyping: Ethnicity-Only vs. Age × Gender × Ethnicity Joint Stratification
- Cross-Ethnic Discrimination AUC Head-to-Head: “Absolute Threshold vs. Within-Group Standardization” Models for Three-Dimensional Phenotyping
- Research Pool 3: Biofrequency Codes
Evidence Level: B
References: PMID 36266618, PMID 33380887, PMID 12812818, PMID 24966219, PMID 27881078, PMID 21411281, PMID 41391734, PMID 37176559, PMID 15286929, PMID 16362542, PMID 38365599, PMID 24449319, PMID 22422832, PMID 15223704
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.