五步联合验证为什么几乎不可能通过:合取判据的乘法代价

Written by

in

Language

Evidence level: B

先说结论

一套”三步、每步四条判据、全过才算通过”的验证链,看起来只是”我们查了 12 项”。但把 12 项当合取(AND)之后,通过率按乘法塌下来:如果每条判据在真值恰好落在门槛上时只有一半机会判对,那么整链通过率是 0.5¹² ≈ 0.024%。即使每条判据有 80% 的正确率,整链通过率也只有 6.9%。

更值得注意的不是”通过很难”,而是”作废很容易”:当同一套结构把”三步中至少三步失败”作为降级条件时,在每条判据毫无分辨力的极端情形下,被判”作废”的概率高达 82%,而被判”通过”的概率只有 0.024%。这两件事方向相反,叠加后体系会单向偏向”作废”。

1. 结构长什么样

这条链(下称 N19 链)由三步组成,每步四条判据,共 12 条:

步 判据(摘要)
站点交叉核对 相关系数 ≥ 0.7;外包络差 ≤ 0.15;变暖贡献 ∈ [30%, 50%];城市差异 ≤ 30%
模式对比 四项 M 判据,形式同上
城市化剥离 趋势差 < 0.05 ℃/decade;相关系数差 ≤ 0.07;占比 ∈ [30%, 41%];两套资料差 ≤ 0.10 ℃/decade

结果处置是步级的,不是判据级:零步失败且上游基座达标 → 升级候选;一至两步失败 → 降一档;三步及以上失败 → 降两档并宣布原结论作废;任一步未启动 → 维持原等级。

2. 乘法代价

设单条判据在”真值恰落门槛”时的通过率为 p,则每步(4 条 AND)的通过率是 p⁴,三步全过是 p¹²:

单条判据通过率 p 单步通过率 p⁴ 三步全过 三步全失败
0.5 6.25% 0.024% 82.40%
0.7 24.01% 1.38% 43.88%
0.8 40.96% 6.87% 20.58%
0.9 65.61% 28.24% 4.07%

这张表的读法是:“查了 12 项”不等于”要求很严”,而是”把通过率压到 0.02%–7%”。如果对外说”我们做了五步联合验证”,读者会以为这是稳健的标志,实际是把一个本来就难的判定压到几乎不可能达成。

3. 被最钝的一道决定

如果四条判据的通过率不同,联合通过率并不是平均,而是被最低的那条主导。以 0.95 / 0.90 / 0.85 / 0.50 为例:

  • 联合 = 36.3%
  • 把最钝的一道从 0.50 提到 0.95 → 联合升到 72.7%,收益 2.00 倍
  • 把任一条从 0.90 提到 0.95 → 收益只有 0.95 倍(几乎为零)

所以”多加几道判据更稳”和”平均提升每条判据”都不是正确动作。正确动作永远是:先找最钝的那一道,只修它。

4. 双向不对称:通过稀薄、作废廉价

上面两张表要一起看。在 p=0.5(判据没有分辨力)时:

判"通过"(三步零失败):0.024%
判"作废"(三步全失败):82.4%

这不是”体系很保守”,而是体系在没信息的时候会大声说”作废”。原因在结构:每步是 4 连 AND,任一条判错整步就算失败;而”失败”与”没做对”在判据层面无法区分。于是精度不足会被自动翻译成”结论不成立”。

这与”止损闸门不该把不可逆侧阈值设在目标值上”是同一条原则的多层放大版:单层时误杀一半,叠加两层后误杀变成单向偏置。

5. 模板表漏了一格

把处置规则穷举,16 种组合里有一种无处可落:三步全部实测通过,但上游基座(三源再分析交叉验证)未达 B+。

这不是假想的边角情形,而是很常见的真实情形——下游三步跑通了,上游数据还没站住。按现行文本,执行者会卡住,然后临时自选一个模板(最可能选”升级候选”,因为”三步都过了”),于是”上游不足”被静默吞掉。

修法是把这一格显式补上:三步零失败但上游未达标 → 维持原等级、不登记发布候选、先补上游。

6. 一个和判据无关但更该先修的问题

这条链引用的中国均一化气温数据集,编号写错了类型。

文中把 11436746 标为 PMID。但把 11436746 送进 PubMed,返回的是一篇 2001 年关于门诊精神科服务满意度的论文;把它送进 PMC 数据库,才返回目标数据集论文(CUG-CMA CHDT,1840–2020 年中国日均温)。也就是说,这是 PMCID 的数字部分被当成了 PMID。

同一篇文献有三个编号,必须分清:

PMID  39333104
PMCID PMC11436746
DOI   10.1038/s41597-024-03758-3

作者也写错了:文中两处写 Cao et al. 2024,实际第一作者是 Zheng X,任国玉为第三作者兼通讯;另有一处写 Xu et al.。

为什么这类错误特别危险:11436746 是一个真实存在的编号,只是指向另一篇论文。任何”检查 PMID 是否存在”的校验都会通过——校验必须比对标题与作者,不能只验存在性。目前这个写法扩散在 9 个文件、22 处,其中同一文件内并存 PMC11436746(对)与 PMID 11436746(错)两种写法,说明是复制粘贴时前缀丢失,而不是理解错误,修起来是机械替换。

另有一处引用(36089010,中国 86 城热岛研究)被用来支撑”站点被城市包围比例从 30% 升到 60%”。该文摘要给出的是热岛强度 0.33 ℃、趋势 0.013 ℃/10a、城乡法贡献 53%、观测减再分析 23%,没有”30%→60% 站点比例”这个数字。该数可能出自正文,但本轮未核到,应标”待核”,不应写成已引。

7. 证据边界

标 B。三条文献均逐字段回查(含 PMCID/DOI 三编号对齐):

  • PMID 39333104 / PMC11436746 / DOI 10.1038/s41597-024-03758-3:Zheng X, Ren Y, Ren G 等,Sci Data 2024;11:1045,中国 1840–2020 日均温均一化数据集。
  • PMID 36089010:Zhou D 等,Sci Total Environ 2023;854:158638,中国 86 城大气热岛多视角研究。
  • PMID 18457487:Meade 等,J Appl Psychol 2008,测量不变性中拟合指标的功效与样本量敏感性。

没有一条直接研究”多层合取判据的联合功效”——那是本项目自建的结构,故只能标 B。

本文所有通过率都建立在”12 条判据相互独立”的假设下。真实情形里,同一步的四条判据往往由同一批站点、同一段序列算出,彼此正相关,实际通过率会高于表中的值。所以本文主张的不是”真实通过率是 6.9%”,而是”必须报联合通过率;不报就等于把严格度藏起来”。

“作废概率 82%”同理,是极端情形(判据完全无分辨力)的上界讨论,不是对 41% 归因结论的任何判断。它既不说 41% 为真,也不说 41% 为假,只说这套判据在现有设置下无法区分”通过了”和”没做对”。

可执行修订

  1. 引用”五步联合验证”时,同时报判据总数、每条预期通过率、联合通过率。
  2. 找出并优先修最钝的那一条判据,不要平均加严。
  3. 模板表补一格:三步零失败但上游基座未达标 → 维持原等级、不登记发布候选。
  4. 全库把 PMID 11436746 改为 PMID 39333104(保留 PMC11436746 写法),作者改为 Zheng/Ren/Ren;Cao et al. 与 Xu et al. 同步更正。
  5. 36089010 支撑的”30%→60%”改标”待核”。
  6. 校验引文时同时比对标题与作者,不只验编号存在。

免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Why Five-Way Joint Verification is Almost Impossible: The Multiplicative Cost of the Conjunction Criterion

Evidence level: B

Conclusion First

A verification chain described as “three steps, four criteria per step, all must pass” appears to be simply “we checked 12 items.” However, treating these 12 items as a conjunction (AND) causes the pass rate to collapse multiplicatively: if each criterion has only a 50% chance of being correctly judged when the true value lies exactly at the threshold, the overall chain pass rate is 0.5¹² ≈ 0.024%. Even if each criterion has an 80% accuracy, the overall pass rate drops to only 6.9%.

What is more noteworthy is not that “passing is hard,” but that “being invalidated is easy”: when the same structure uses “failure in at least three of the three steps” as a downgrade condition, the probability of being judged “invalidated” reaches 82% in the extreme case where each criterion has no discriminative power, while the probability of being judged “passed” is only 0.024%. These two directions oppose each other, causing the system to unidirectionally bias toward “invalidation.”

1. What Does the Structure Look Like?

This chain (hereafter N19 chain) consists of three steps, each with four criteria, totaling 12:

Step Criteria (Summary)
Site Cross-Check Correlation coefficient ≥ 0.7; outer envelope difference ≤ 0.15; warming contribution ∈ [30%, 50%]; urban difference ≤ 30%
Pattern Comparison Four M criteria, form as above
Urbanization Removal Trend difference < 0.05 ℃/decade; correlation coefficient difference ≤ 0.07; proportion ∈ [30%, 41%]; two datasets difference ≤ 0.10 ℃/decade

Result handling is step-level, not criterion-level: zero step failures with upstream base达标 → upgrade candidate; one or two step failures → downgrade one tier; three or more step failures → downgrade two tiers and declare original conclusion invalid; any step not initiated → maintain original tier.

2. Multiplicative Cost

Let the pass rate of a single criterion when “the true value lies exactly at the threshold” be p. Then the pass rate of each step (4 AND) is p⁴, and passing all three steps is p¹²:

Single Criterion Pass Rate p Single Step Pass Rate p⁴ All Three Steps Pass All Three Steps Fail
0.5 6.25% 0.024% 82.40%
0.7 24.01% 1.38% 43.88%
0.8 40.96% 6.87% 20.58%
0.9 65.61% 28.24% 4.07%

The reading of this table is: “Checking 12 items” does not mean “requiring strictness,” but rather “compressing the pass rate to 0.02%–7%.” If externally stating “we performed five-step joint verification,” readers will think this is a sign of robustness, actually compressing an already difficult judgment into almost impossible to achieve.

3. Decided by the Dullest Criterion

If the pass rates of the four criteria differ, the joint pass rate is not averaged but dominated by the lowest one. Taking 0.95 / 0.90 / 0.85 / 0.50 as an example:

  • Joint = 36.3%
  • Raising the dullest criterion from 0.50 to 0.95 → joint rises to 72.7%, gain 2.00 times
  • Raising any criterion from 0.90 to 0.95 → gain only 0.95 times (almost zero)

Therefore “adding more criteria makes it more stable” and “averaging improvement of each criterion” are not correct actions. The correct action is always: first find the dullest one, fix only it.

4. Bidirectional Asymmetry: Passing Thin, Invalidating Cheap

The above two tables must be read together. At p=0.5 (criteria have no discriminative power):

Judge "passed" (zero failures in three steps): 0.024%
Judge "invalidated" (all three steps fail): 82.4%

This is not “the system is very conservative,” but rather the system loudly says “invalidated” when there is no information. The reason lies in the structure: each step is 4-con AND, any single criterion failure makes the whole step fail; while “failure” and “not done right” cannot be distinguished at the criterion level. Thus insufficient precision is automatically translated into “conclusion not valid.”

This is a multi-layer amplification of the same principle as “the stop-loss gate should not set the irreversible-side threshold at the target value”: single-layer mis-kills half, adding two layers makes mis-kill become unidirectional bias.

5. Template Table Missed One Cell

Exhausting the disposal rules, among 16 combinations there is one nowhere to fall: all three steps pass in measurement, but the upstream base (three-source reanalysis cross-validation) does not reach B+.

This is not a hypothetical corner case, but a very common real situation—downstream three steps run through, upstream data not yet standing. According to current text, the executor will be stuck, then temporarily choose a template (most likely “upgrade candidate,” because “all three passed”), thus “upstream insufficient” is silently swallowed.

The fix is to explicitly add this cell: three steps zero failures but upstream not met → maintain original tier, do not register publish candidate, first supplement upstream.

6. One Issue Unrelated to Criteria but More Urgent to Fix

The chain referenced the Chinese homogenized temperature dataset, with the number written wrong type.

In the text 11436746 is marked as PMID. But sending 11436746 into PubMed returns an article from 2001 about outpatient psychiatry service satisfaction; sending it into PMC database returns the target dataset article (CUG-CMA CHDT, 1840–2020 China daily mean temperature). That is, this is the PMCID number part mistakenly treated as PMID.

The same article has three numbers, must distinguish:

PMID  39333104
PMCID PMC11436746
DOI   10.1038/s41597-024-03758-3

The authors also wrote wrong: two places write Cao et al. 2024, actually first author is Zheng X, Ren Guoyu is third author and corresponding; another place writes Xu et al..

Why such errors are especially dangerous: 11436746 is a real existing number, just pointing to another paper. Any “check whether PMID exists” validation will pass—validation must compare title and authors, not only existence. Currently this writing spreads in 9 files, 22 places, where within the same file both PMC11436746 (correct) and PMID 11436746 (wrong) coexist, indicating copy-paste lost prefix rather than understanding error, fixing it is mechanical replacement.

Another citation (36089010, China 86-city heat island study) is used to support “site surrounded by urban proportion rose from 30% to 60%”. The abstract of that paper gives heat island intensity 0.33 ℃, trend 0.013 ℃/10a, urban-rural method contribution 53%, observation minus reanalysis 23%, no “30%→60% site proportion” number. This number may come from the main text, but not checked in this round, should mark “pending verification,” not written as cited.

7. Evidence Boundary

Mark B. Three literature sources checked field-by-field (including PMCID/DOI three-number alignment):

  • PMID 39333104 / PMC11436746 / DOI 10.1038/s41597-024-03758-3: Zheng X, Ren Y, Ren G et al., Sci Data 2024;11:1045, China 1840–2020 daily mean temperature homogenized dataset.
  • PMID 36089010: Zhou D et al., Sci Total Environ 2023;854:158638, China 86-city atmospheric heat island multi-perspective study.
  • PMID 18457487: Meade et al., J Appl Psychol 2008, power and sample size sensitivity of fit indices in measurement invariance.

None directly studies “joint efficacy of multi-layer conjunction criteria”—that is the structure built by this project, so can only mark B.

All pass rates in this paper are based on the assumption that “12 criteria are mutually independent.” In reality, the four criteria of each step are often calculated from the same set of sites and same sequence, positively correlated, actual pass rate will be higher than values in the table. Therefore this paper does not claim “real pass rate is 6.9%,” but rather “must report joint pass rate; not reporting equals hiding strictness.”

“Invalidation probability 82%” similarly, is the upper bound discussion of extreme case (criteria completely non-discriminative), not any judgment on the 41% attribution conclusion. It neither says 41% is true nor false, only that under current settings this set of criteria cannot distinguish “passed” and “not done right.”

Executable Revisions

  1. When citing “five-step joint verification,” simultaneously report total number of criteria, expected pass rate per criterion, joint pass rate.
  2. Find and prioritize fixing the dullest criterion, do not average tightening.
  3. Template table add one cell: three steps zero failures but upstream base not met → maintain original tier, do not register publish candidate.
  4. Change PMID 11436746 to PMID 39333104 throughout the entire database (keep PMC11436746 writing), authors corrected to Zheng/Ren/Ren; Cao et al. and Xu et al. simultaneously corrected.
  5. The “30%→60%” supported by 36089010 marked “pending verification.”
  6. When validating citations, simultaneously compare title and authors, not only check number existence.

Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *