换个县还灵不灵:跨地质背景验证前,得先摆上桌的三个数

Written by

in

Language

Evidence level: B

跨区验证最容易犯的错,是把”性能掉多少”当成问题;真问题是——那个被验证的”增量”,在另一个地质背景里还是不是同一个量。

正文

一个县里做出来的凶地评级增量,拿到花岗岩区、碳酸盐区、平原区各验一遍,看 ΔAUC 的符号是否一致——这个设计听起来很干净。但它跳过了三个更靠前的数。

第一层,外部验证的门槛不是点数,是事件数。 34031906 给出二分类结局外部验证的最小样本量算法:目标是让校准(O/E 比与校准斜率)、判别(C 统计量)、临床净收益三项都达到预设精度。示例里事件率 0.018,算出来要 9835 人、177 个事件,而且这个数字是被”校准斜率”这一条顶上去的。33150684 是连续结局版本,四条准则(R²、calibration-in-the-large、校准斜率、结局方差)取最大者。回到凶地评级:一个县四十个点,超标率按 15% 算,事件数是 6 个。六个事件连校准斜率的区间都撑不开,ΔAUC 的区间宽度会盖住整个可能的效应范围。这不是验证失败,是没在验证。

第二层,外部验证的先验是掉,不是涨。 25441703 复查 127 个新提出的风险模型:只有 32 个(25%)找得到至少一次外部验证,其中 22 个由完全不同的作者完成;AUC 相对推导中位下降 0.05,19 个模型降幅达到或超过 0.03,没有一个上升达到 0.03。所以判据不能写成”跨区 ΔAUC 显著为正”,得写成”跌幅落在 0 到 0.05 之间”。前者在要求数据推翻一个已知规律。

第三层,判别与校准是两条独立的命。 36378565 把 43 个重症死亡模型拿去两个独立队列验证,只有 11 个(26%)能重建;能验的那几个,AUC 在 0.63 到 0.83 之间、可接受至优秀,但校准普遍差,重校准截距与斜率后才到中等。31842878 说得更直:校准是预测分析的阿喀琉斯之踵。跨区”方向一致”若只盯 AUC,漏掉的正是最要命的一半:同一个分数在两县代表的绝对风险完全不同。

第四层,可分配的空间本身是地区属性。 19577346 用方差分析给出硬数字:英格兰与威尔士,填图基岩与表土地质解释室内氡总变异的约 25%;在沉积地质单元氡潜势反差强的地区升到 37%;而在铀矿化这类混杂控制横切地质界线的地区只有 14%。同一个国家、同一套变量、同一个方法,解释力从 14% 到 37%,差 2.6 倍。这意味着”评级还能再加多少”的上限在两地差几倍:ΔR²=0.04 在 37% 的区是锦上添花,在 14% 的区可能是仅剩的空间。跨区比 ΔR² 的大小,等于拿两把不同刻度的尺子量同一件事。

第五层,变量重要性也随区漂移。 34863751 在意大利 Euganean Hills 用回归克里金建地生氡潜势图,结论是潜势主要由基岩类型驱动,但断层密度与地形控制氡在土体中的运移与地表呼出;33254926 在德国用 4448 个实测点加 40 km 空间块交叉验证,也是地质变量主导。两句合起来看:哪个变量”最重要”是区属性。于是同一个凶地评级,在 A 区可能是断层的粗代理,在 B 区是岩性的粗代理——符号一致或翻转,测的其实是”评级在两区替谁说话”。

第六层,判一致性的方法不是分区各跑一遍。 11701102 用仿真量化过:在真实无差异的数据上,正式交互检验 5% 假阳性(正确),而按亚组分别做显著性判断会在 7% 到 66% 的情形下”发现”差异,且常报错差异的类型;交互检验功效普遍更好,但要稳定检出,交互大小需达到总效应两倍左右。正确结构是:一个模型加区×评级交互项,配 ICEMAN 评可信度(32778601),用 GRADE 36 区分”结果变异”与”研究特征变异”(36898507)。而交互检验的功效要求反过来说明:四个区各四十点,这个设计无论数据长什么样都得不出”方向一致”。

第七层,若系数本身就是空间非平稳的,”一致”这个前提就错了。 41873991 用多尺度地理加权回归在全国尺度上看到显著的空间非平稳,全局模型把局部的系数差异抹平。所以该问的不是”是否处处一致”,而是”在哪个尺度上可迁移”。出路有一条实证:35614485 在多个医疗数据库间做迁移,融合集成比单库模型判别更好、跨库更一致,但校准在未见数据上一致地差,必须用新数据重校准。能搬的是排序,不能搬的是水平。

落到判决上,四件事:① 先算事件数,按校准斜率反推每区最小样本量,达不到只做区内不下跨区结论;② 判据写成”跌幅在 0 至 0.05 之间”,不写”显著为正”;③ 报判别必报校准,两区各给校准曲线而不是只给一个 AUC 差;④ 一致性用”区×评级”交互项加 ICEMAN 打分,不用分区各跑显著性。

易学这一侧,古人的规矩恰好是对着这件事说的。形峦派讲「一方山水一方龙」,又讲看地要「先看祖宗、再看父母」——祖宗是发脉的高山、父母是结穴的近身之山,同一条龙从祖宗到父母气脉连续,换个县的另一条山脉,祖宗父母全换,走法不能照搬。这不是保守,是经验上早发现了”同一套标准跨不过山脉”。更具体的落点是「山川有异,风气不同」——古书论形势从不给全国通用刻度,要求就地取材、随龙定穴。用统计话说:古人不假设系数平稳,只假设判据的结构(龙、穴、砂、水、向这五个位置有先后与主从)平稳。这正好对上第七层:能搬的是顺序,不能搬的是权重。

证据与局限

证据与局限

已验证: 外部验证的最小样本量由校准精度主导,事件数而非点数决定可行性(34031906、33150684);外部验证的判别力普遍下降、几乎从不上升(25441703);判别可保持而校准常崩,重校准可部分修复(36378565、31842878);地质变量对室内氡的解释力在同一国家内从 14% 到 37% 不等(19577346);亚组内部分析的假阳性远高于交互检验(11701102);空间非平稳在区域尺度上普遍存在(41873991)。

部分验证: 跨库集成可提升迁移后的判别一致性,但校准仍需重校(35614485);不同地区驱动变量排序不同(34863751、33254926)。

待验证: 凶地评级的增量在多地质背景间是否方向一致;”可分配空间”差异是否足以解释跨区增量幅度的差异;把”局”操作化为可编码层之后,交互项能否检出。

相关阅读

证据等级: B(外部验证方法学、校准失效、地质解释力跨区差异均有 PubMed 直接支撑;”凶地评级增量的跨区方向一致性”本身无先例,属研究设计提案)
参考文献: PMID 34031906, PMID 33150684, PMID 25441703, PMID 36378565, PMID 31842878, PMID 19577346, PMID 34863751, PMID 33254926, PMID 11701102, PMID 32778601, PMID 36898507, PMID 41873991, PMID 35614485


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Does it still work in a different county: Three numbers must be laid on the table before validating across geological backgrounds

Evidence level: B

The most common error in cross-regional validation is treating “how much performance drops” as the problem; the real issue is whether the validated “increment” remains the same quantity in a different geological background.

Main Text

An incremental risk rating derived from a single county, applied to granite, carbonate, and plain regions to check if the sign of ΔAUC remains consistent—this design sounds clean. However, it overlooks three more fundamental numbers.

First Layer: The threshold for external validation is not sample size, but event count. 34031906 provides an algorithm for the minimum sample size required for external validation in binary outcomes: the goal is to ensure calibration (O/E ratio vs. calibration slope), discrimination (C-statistic), and clinical net benefit all meet preset precision levels. In the example, with an event rate of 0.018, the calculation yields 9,835 people and 177 events; this number is driven upward by the “calibration slope” criterion. 33150684 addresses continuous outcomes, using the maximum of four criteria (R², calibration-in-the-large, calibration slope, outcome variance). Returning to risk rating: a county with 40 points and an overage rate of 15% yields only 6 events. Six events cannot even span the interval for the calibration slope; the width of the ΔAUC interval would swallow the entire possible effect range. This is not a validation failure; it is a failure to validate.

Second Layer: The prior for external validation is decline, not increase. 25441703 reviewed 127 newly proposed risk models: only 32 (25%) found at least one external validation, of which 22 were completed by authors entirely different from the original; AUC relative to the derivation median dropped by 0.05, and in 19 models the drop reached or exceeded 0.03, with none rising by 0.03. Therefore, the criterion cannot be written as “cross-regional ΔAUC significantly positive”; it must be written as “drop falls between 0 and 0.05.” The former demands data overturn a known pattern.

Third Layer: Discrimination and calibration are two separate lives. 36378565 took 43 severe mortality models to two independent cohorts for validation; only 11 (26%) could be reconstructed. Among those validated, AUC ranged from 0.63 to 0.83 (acceptable to excellent), but calibration was generally poor, reaching only moderate levels after recalibrating intercept and slope. 31842878 states it more directly: calibration is the Achilles’ heel of predictive analysis. If cross-regional “directional consistency” focuses only on AUC, it misses the most critical half: the same score represents vastly different absolute risks in two counties.

Fourth Layer: The allocatable space itself is a regional attribute. 19577346 used ANOVA to provide hard numbers: in England and Wales, mapping bedrock versus surface geology explained approximately 25% of the total variance in indoor radon; in sedimentary geological units with strong radon potential contrast, this rose to 37%; whereas in areas with uranium mineralization where mixed controls cross-cut geological boundaries, it was only 14%. Same country, same variables, same method—explanatory power varied from 14% to 37%, a 2.6-fold difference. This means the ceiling for “how much more rating can be added” differs by several times between regions: a ΔR² of 0.04 is icing on the cake in the 37% region but may represent the remaining space in the 14% region. Comparing cross-regional ΔR² magnitudes is like measuring the same thing with two rulers of different scales.

Fifth Layer: Variable importance also drifts by region. 34863751 built a geogenic radon potential map using regression kriging in the Euganean Hills, Italy, concluding that potential is primarily driven by bedrock type, while fault density and topography control radon migration in soil and surface exhalation; 33254926 used 4,448 measured points plus 40 km spatial blocks for cross-validation in Germany, also finding geological variables dominant. Taken together: which variable is “most important” is a regional attribute. Thus, the same risk rating in Region A might be a coarse proxy for faults, while in Region B it is a coarse proxy for lithology—whether the sign remains consistent or flips, what is actually measured is “who the rating speaks for in each region.”

Sixth Layer: The method to judge consistency is not running separate analyses per region. 11701102 quantified this via simulation: on truly null data, formal interaction testing yields a 5% false positive rate (correct), whereas performing significance tests separately by subgroup would “discover” differences in 7% to 66% of cases, often misreporting the type of difference. Interaction testing generally has better power, but to stably detect it, interaction size needs to be roughly double the main effect. The correct structure is: one model adding a region × rating interaction term, assessed for credibility using ICEMAN (32778601), and distinguishing “outcome heterogeneity” from “study characteristic heterogeneity” using GRADE 36 (36898507). The power requirements for interaction testing conversely demonstrate that with four regions of 40 points each, this design will never yield “directional consistency” regardless of data appearance.

Seventh Layer: If coefficients themselves are spatially non-stationary, the premise of “consistency” is wrong. 41873991 observed significant spatial non-stationarity at a national scale using multi-scale geographically weighted regression, where global models flatten local coefficient differences. Therefore, the question should not be “is it consistent everywhere,” but “at what scale is it transferable.” One empirical path exists: 35614485 performed migration across multiple medical databases, finding that ensemble models showed better discrimination and cross-database consistency than single-database models, but calibration was consistently poor on unseen data and required recalibration with new data. What can be transferred is ranking; what cannot is level.

In terms of judgment, four things must be done: ① First calculate event counts and back-calculate minimum sample sizes per region based on calibration slope; if not met, do not draw cross-regional conclusions; ② Write the criterion as “drop falls between 0 and 0.05,” not “significantly positive”; ③ Report both discrimination and calibration, providing calibration curves for each region rather than just a single ΔAUC; ④ Use “region × rating” interaction terms with ICEMAN scoring, avoiding separate significance tests per region.

On the Yijing Studies side, ancient rules align perfectly with this matter. The Form School speaks of “One Landscape, One Dragon” and notes that site selection requires “Looking at Ancestors First, Then Parents“—Ancestors are the high mountains initiating the lineage, Parents are the close-range mountains forming the tomb site; if the dragon’s energy flows continuously from Ancestors to Parents, changing counties means a different mountain range where both Ancestors and Parents change, so the method cannot be copied blindly. This is not conservatism; it is an empirical realization that “a single set of standards cannot cross mountain ranges.” The more specific application is “Different Mountains, Different Winds“—ancient texts on form never provided a universal scale for the nation, demanding local materials and determining the tomb site following the dragon’s path. In statistical terms: ancients did not assume coefficient stationarity; they only assumed the structure of the criterion (the sequence and hierarchy of Dragon, Tomb, Sand, Water, and Direction) was stationary. This aligns perfectly with Layer Seven: what can be transferred is the sequence; what cannot is the weight.

Evidence and Limitations

Evidence and Limitations

Verified: Minimum sample size for external validation is driven by calibration precision, determined by event count rather than point count (34031906, 33150684); discrimination generally declines in external validation with almost no increase (25441703); discrimination can be maintained while calibration often collapses, partially repairable via recalibration (36378565, 31842878); explanatory power of geological variables for indoor radon varies within the same country from 14% to 37% (19577346); subgroup internal analysis yields far higher false positives than interaction testing (11701102); spatial non-stationarity is common at regional scales (41873991).

Partially Verified: Cross-database ensembles can improve post-migration discrimination consistency, but calibration still requires recalibration (35614485); driving variable rankings differ across regions (34863751, 33254926).

To Be Verified: Whether the incremental risk rating maintains directional consistency across multiple geological backgrounds; whether differences in “allocatable space” sufficiently explain variations in cross-regional increment magnitude; whether interaction terms can detect effects after operationalizing the “local” as an encoded layer.

Related Reading

Evidence Level: B (External validation methodology, calibration failure, and cross-regional differences in geological explanatory power all have direct PubMed support; “cross-regional directional consistency of risk rating increments” itself has no precedent, belonging to a research design proposal)
References: PMID 34031906, PMID 33150684, PMID 25441703, PMID 36378565, PMID 31842878, PMID 19577346, PMID 34863751, PMID 33254926, PMID 11701102, PMID 32778601, PMID 36898507, PMID 41873991, PMID 35614485


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *