命盘评分加入临床风险模型后的增量检验:NRI 与净获益能给出什么答案

Written by

in

Language

Evidence level: B

把”命盘评分”当候选变量塞进临床风险模型,检验它能否在家族史与多基因评分之上带来判别增量——这个检验真正难的不是命盘灵不灵,而是”用哪把尺子量”。三把尺子会给出三个方向相反的答案。

正文

上一篇我们做了三方头对头:同一队列里,出生时刻(时辰粒度)、家族史、多基因风险评分(PRS)谁更能判别个体。这一篇要问的是更靠后的一步——如果真把”命盘评分”操作化成一个连续变量,加进一个已经在用的临床风险模型(比如含年龄、家族史、PRS 的基线模型),该怎么判断它有没有带来”增量”?

答案取决于尺子。 医学界为此打了近二十年的仗,结论是:这三把常用尺子会互相打架。

第一把尺子:判别力增量(ΔAUC / ΔC-index)。 它最直觉,却对强预测因子异常迟钝。Steyerberg 等人(2012)用睾丸癌残留肿瘤 544 例示范:AUC 和 R² 推荐加入 LDH 与 AFP,但按 20% 临床决策阈值算净获益,只有 HCG 站得住;NRI 却认为三个标志物都有”重分类潜力”(PMID 21726217)——同一份数据,三把尺子三个赢家。原因不是尺子坏了,而是”统计显著”≠”临床有用”:P 值不衡量效应大小(PMID 31560143)。

那么”小增量”到底算不算有价值?Baker 等人(2014)给了个漂亮的解法:加产时标志物让 AUC 只涨 0.02,看着可怜,但按决策分析换算成”每识别 1 例正确预测需要采集 68 到 124 人的数据”——数据采集无创,这个代价可以接受,所以 0.02 是有价值的(PMID 24825728)。关键点:ΔAUC 本身无法回答”够不够”,只有接上后果才行。

第二把尺子:净重分类指数(NRI)。 它最贴合”重分类”直觉,问题也最大。Pepe 等人(2014)做了个残酷模拟:10,000 人、事件率 10.2%、四个完全无预测力的标志物反复模拟,NRI 给出”正且显著”的假阳性率在训练集高达 63.0%,独立测试集仍有 18.8%–34.4%;同期 ΔAUC 假阳性率极低,似然比检验稳定在约 5%(PMID 24681599)。他们直言”科学报告中应停止使用 NRI 的 P 值”。Pepe 另一篇(2015)进一步证明:即使采用独立验证集这一最佳设计,NRI 仍会对无用标志物给出正结论——因为它”不恰当”(improper)(PMID 26504496)。

Kerr 等人(2014)的评述划出了可操作边界:两分类时 NRI 的分量就是真阳性率与假阳性率的变化,直接报这两个率更好;三类及以上分类不建议用 NRI,因为它无法恰当处理”跨几个类别的移动”这类临床重要差异;连续 NRI 会高估增量价值,即使在独立验证数据里;若一定要用,置信区间须用 bootstrap 而非已发表方差公式;增量价值的首选单一汇总指标是净获益的改善(PMID 24240655)。三分类 NRI 研究也确认:用这把尺子时,本来就表现好的模型更难被改进(PMID 25176621)。

第三把尺子:净获益与决策曲线。 唯一与临床后果直接挂钩。Vickers 与 Elkin(2006)提出决策曲线分析:把”愿意接受治疗的阈值概率”作为假阴与假阳相对代价的显式表达,画出净获益随阈值变化的曲线,从而回答”这个模型值不值得用、哪个更好”,而不只是”准不准”(PMID 17099194)。当目的是支持决策时,净获益优于 AUC 的整体判断(PMID 21726217)。

这三把尺子在真实数据上怎么表现? 有两个现成的锚点。UK Biobank 30.7 万人(Sun 等,2021):常规模型 C-index 0.710,加入 PRS 后只涨 0.012;换算成群体效果是——全人群筛查每 5,750 人可多防 1 例心血管事件,只在中间风险段(5%–10%)筛查则每 340 人可多防 1 例,比常规预测多防 7%(PMID 33444330)。2025 年欧洲心脏杂志的 UK Biobank 研究(29.7 万人)更极端:SCORE2 基线 C-index 0.719,加 11 个临床标志物涨 0.014、加 NMR 代谢组评分涨 0.010、加 PRS 涨 0.009,三者全加才涨 0.024;对应分类净重分类中净病例重分类 16.66%,群体建模显示每 10 万人筛查可多防 184 例(PMID 41392353)。

最值得命理检验者注意的,是中国人群的负面结果。 中国慢性病前瞻性研究(CKB,Yang 等 2023)在 7.2 万测试集中验证冠心病 PRS:加进传统风险模型后,男性 C-index 只涨 0.003,女性只涨 0.001(置信区间含零);在 1%–10% 各个高风险阈值下,最高的分类 NRI 也只有 3.2%(PMID 37200020)。作者结论很直白:当前 PRS 不适合在中国一般人群中推广基因筛查来改善冠心病风险预测。 ——连经过全基因组数十万样本锤炼的 PRS,在跨人群后都可能落到”几乎无增量”,这个尺度感对任何声称能判别个体命运的体系都是一道硬门槛。

还有两个必须先跨过的前置问题。 其一,模型在开发集上好看不等于真有用:加法总会让表观性能上升,必须做 bootstrap 校正乐观度(PMID 36829188 也提醒:由于人群、测量、时间三重变异,没有真正”已验证”的预测模型,一次外部验证的好结果不构成普遍适用性证明)。其二,ΔAUC、NRI、IDI 在比较嵌套模型的原假设下不服从渐近正态,此时常规标准误公式与 bootstrap 都要重新审视(PMID 28627112);IDI 更被证明标准误被低估约 2 倍、z 检验无效(PMID 21673124),还会因协变量在事件/非事件间不平衡而出现辛普森悖论(PMID 29160558)。

所以对命理体系的检验含义是什么? 把命盘评分当候选变量加进临床模型这件事,在程序上完全可行——本文把程序写清楚了。但必须事前定死四件事:① 用哪把尺子做主判据(建议净获益,附代价比理由,ΔAUC 与 NRI 作辅);② 命盘评分如何操作化(不能事后再选算法);③ 外部验证与 bootstrap 校正乐观度;④ 若增量不显著,停止条件是什么。缺任何一件,测出来的都不是命盘的增量,而是尺子的性质。

证据与局限

已验证: 三把增量尺子会给出方向不一致的结论(PMID 21726217, 24240655);NRI 在独立验证集上仍会对无用标志物给出正结论,假阳性率训练集 63.0%、测试集 18.8%–34.4%(PMID 24681599, 26504496);三类及以上分类不建议用 NRI,首选净获益(PMID 24240655, 25176621);决策曲线分析是与后果挂钩的评估框架(PMID 17099194);真实数据中 PRS 加入常规模型的 ΔC-index 普遍只有 0.009–0.012,中国人群 CAD PRS 仅 0.001–0.003、最高分类 NRI 3.2%(PMID 33444330, 41392353, 37200020)。

部分验证: ΔAUC 虽小但可换算为可接受的”检验代价”(68–124 人/正确预测,PMID 24825728),说明”小增量”不等于无价值——但该换算依赖具体场景的代价比,无法直接搬到命理场景;ΔAUC/NRI/IDI 在原假设下的渐近分布不标准(PMID 28627112)。

待验证: ① 命盘评分如何操作化才能避免”事后选算法”——目前无任何中文命理评分被公开、预注册并做过多队列外部验证;② 在净获益框架下,命盘评分需要多大增量才够”值得采集”,代价比如何设定,无先例;③ 命盘评分与传统变量(出生季节/时刻)的共线性程度,从未量化。

相关阅读

证据等级: B
参考文献: PMID 21726217, PMID 24825728, PMID 17099194, PMID 24240655, PMID 26504496, PMID 24681599, PMID 25176621, PMID 21673124, PMID 29160558, PMID 31560143, PMID 36829188, PMID 33444330, PMID 41392353, PMID 37200020, PMID 28627112


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Incremental Value of Adding Prognostic Score to Clinical Risk Model: What Answers Can NRI and Net Benefit Provide

Evidence level: B

Introducing a “Fate Chart Score” as a candidate variable into a clinical risk model to test whether it provides discriminative increment beyond family history and polygenic risk scores (PRS)—this validation is not primarily about whether the chart is “divinatory,” but rather about “which ruler to use.” Three rulers yield three answers pointing in opposite directions.

Main Text

In the previous article, we conducted a head-to-head comparison within the same cohort: which factor—birth time (hourly granularity), family history, or polygenic risk scores (PRS)—offers superior discrimination for individuals. This article addresses the subsequent step: if we operationalize the “Fate Chart Score” as a continuous variable and integrate it into an existing clinical risk model (e.g., a baseline model containing age, family history, and PRS), how do we determine if it provides an “increment”?

The answer depends on the ruler. The medical community has fought this battle for nearly two decades, concluding that these three common rulers often conflict.

Ruler 1: Discrimination Increment (ΔAUC / ΔC-index). It is the most intuitive yet abnormally insensitive to strong predictors. Steyerberg et al. (2012) demonstrated this using 544 cases of residual testicular cancer: AUC and R² recommended adding LDH and AFP, but calculating net benefit at a 20% clinical decision threshold showed only HCG was robust; NRI suggested all three markers had “reclassification potential” (PMID 21726217)—three winners from the same dataset with three different rulers. The issue is not a broken ruler, but that “statistical significance” ≠ “clinical utility”: P-values do not measure effect size (PMID 31560143).

So, does a “small increment” hold value? Baker et al. (2014) provided an elegant solution: adding perinatal markers increased AUC by only 0.02, which appears negligible, but when converted via decision analysis into “the number of people needed to correctly identify one case,” the requirement was 68 to 124 individuals. Since data collection is non-invasive, this cost is acceptable; thus, 0.02 is valuable (PMID 24825728). Key point: ΔAUC alone cannot answer “is it sufficient”; only when coupled with consequences does it matter.

Ruler 2: Net Reclassification Index (NRI). It aligns best with the intuition of “reclassification,” but carries the largest problems. Pepe et al. (2014) conducted a harsh simulation: with 10,000 individuals, an event rate of 10.2%, and four completely non-predictive markers repeatedly simulated, NRI yielded a false-positive rate of “positive and significant” in the training set as high as 63.0%, with independent test sets showing 18.8%–34.4%; concurrently, ΔAUC showed extremely low false-positive rates, and likelihood ratio tests remained stable at approximately 5% (PMID 24681599). They explicitly stated “P-values for NRI should be stopped in scientific reporting.” Pepe’s subsequent work (2015) further proved that even with independent validation sets—the best design—NRI still yields positive conclusions for useless markers because it is “improper” (PMID 26504496).

Kerr et al. (2014) outlined actionable boundaries in their commentary: for binary classification, NRI components represent changes in true-positive and false-positive rates; reporting these rates directly is better; NRI is not recommended for three or more categories as it cannot appropriately handle clinically important differences involving movement across multiple categories; continuous NRI overestimates incremental value even in independent validation data; if NRI must be used, confidence intervals should employ bootstrap rather than published variance formulas; the preferred single summary metric for incremental value is improvement in net benefit (PMID 24240655). Three-category NRI studies also confirmed: models already performing well are harder to improve using this ruler (PMID 251736621).

Ruler 3: Net Benefit and Decision Curves. The only one directly linked to clinical consequences. Vickers and Elkin (2006) proposed decision curve analysis: expressing the “threshold probability willing to accept treatment” as an explicit trade-off between false-negative and false-positive costs, plotting net benefit against threshold probability to answer “is this model worth using, which is better,” rather than just “how accurate” (PMID 17099194). When the goal is to support decision-making, net benefit outperforms AUC for overall judgment (PMID 21726217).

How do these three rulers perform on real data? There are two ready-made anchors. In the UK Biobank with 307,000 individuals (Sun et al., 2021): the conventional model had a C-index of 0.710; adding PRS increased it by only 0.012; converting to population effect means screening the entire population prevents 1 cardiovascular event for every 5,750 screened, while screening only the intermediate-risk segment (5%–10%) prevents 1 event for every 340 screened, a 7% improvement over conventional prediction (PMID 33444330). A 2025 European Heart Journal study on UK Biobank (297,000 individuals) was more extreme: baseline SCORE2 C-index 0.719; adding 11 clinical markers increased it by 0.014, adding NMR metabolomic scores by 0.010, adding PRS by 0.009; all three together increased it by only 0.024; corresponding classification net reclassification showed 16.66% net case reclassification, and population modeling indicated screening every 100,000 could prevent 184 additional cases (PMID 41392353).

The result most significant for fate chart validators is the negative finding in Chinese populations. The China Kadoorie Biobank (CKB, Yang et al. 2023) validated CAD PRS in a 72,000-person test set: adding to traditional risk models increased C-index by only 0.003 for males and 0.001 for females (confidence intervals included zero); across 1%–10% high-risk thresholds, the highest classification NRI was only 3.2% (PMID 37200020). The authors’ conclusion is straightforward: Current PRS is not suitable for promoting gene screening in the general Chinese population to improve CAD risk prediction. —Even PRS refined by hundreds of thousands of genome-wide samples can fall to “almost no increment” after cross-population transfer; this sense of scale is a hard threshold for any system claiming to discriminate individual fate.

There are two prerequisite issues that must be crossed first. First, good appearance in the development set does not equal true utility: addition always inflates apparent performance; bootstrap correction for optimism is mandatory (PMID 36829188 also reminds: due to triple variation in population, measurement, and time, no truly “validated” predictive model exists; a good result from a single external validation does not constitute proof of generalizability). Second, under the null hypothesis comparing nested models, ΔAUC, NRI, and IDI do not follow asymptotic normality; thus conventional standard error formulas and bootstrap require re-examination (PMID 28627112); IDI has been proven to underestimate standard errors by approximately 2-fold, rendering z-tests invalid (PMID 21673124), and may also exhibit Simpson’s paradox due to imbalance of covariates between event and non-event groups (PMID 29160558).

So, what is the implication for validating fate systems? Adding a Fate Chart Score as a candidate variable into a clinical model is procedurally fully feasible—this article clarifies the procedure. However, four things must be fixed beforehand: ① Which ruler serves as the primary criterion (recommend Net Benefit with cost-benefit rationale, using ΔAUC and NRI as auxiliaries); ② How the Fate Chart Score is operationalized (cannot select algorithms post-hoc); ③ External validation and bootstrap correction for optimism; ④ What constitutes the stopping condition if the increment is non-significant. Missing any one means the measured result is not the chart’s increment, but the nature of the ruler.

Evidence and Limitations

Verified: Three incremental rulers yield inconsistent directional conclusions (PMID 21726217, 24240655); NRI yields positive conclusions for useless markers in independent validation sets with false-positive rates of 63.0% in training and 18.8%–34.4% in testing (PMID 24681599, 26504496); NRI is not recommended for three or more categories; net benefit is preferred (PMID 24240655, 251736621); decision curve analysis is a consequence-linked evaluation framework (PMID 17099194); in real data, ΔC-index for adding PRS to conventional models is generally 0.009–0.012, with Chinese population CAD PRS at only 0.001–0.003 and highest classification NRI at 3.2% (PMID 33444330, 41392353, 37200020).

Partially Verified: ΔAUC is small but convertible to acceptable “testing costs” (68–124 people per correct prediction, PMID 24825728), indicating “small increment” does not equal no value—but this conversion relies on specific scenario cost ratios and cannot be directly transferred to fate contexts; ΔAUC/NRI/IDI do not follow standard asymptotic distributions under the null hypothesis (PMID 28627112).

To Be Verified: ① How to operationalize the Fate Chart Score to avoid “post-hoc algorithm selection”—currently no Chinese fate scoring system has been publicly disclosed, preregistered, and validated across multiple external cohorts; ② Within the net benefit framework, what magnitude of increment is required for the Fate Chart Score to be deemed “worth collecting,” and how cost ratios should be set—no precedents exist; ③ The degree of collinearity between Fate Chart Scores and traditional variables (birth season/time) has never been quantified.

Related Reading

Evidence Level: B
References: PMID 21726217, PMID 24825728, PMID 17099194, PMID 24240655, PMID 26504496, PMID 24681599, PMID 251736621, PMID 21673124, PMID 29160558, PMID 31560143, PMID 36829188, PMID 33444330, PMID 41392353, PMID 37200020, PMID 28627112


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *