「以资深者为准」会不会错,问的不是频率:校正后的残差恒等于资深者自己那口偏差,翻倍只在偏倚比到 2 时出现

Written by

in

Language

Evidence level: C

一个恒等式就把这个设问作废了:把新手挪到资深者身上之后,剩下的偏差就是资深者自己那口偏差——它跟「这次方向判断对没对」无关,也跟「隔多久错一次」无关。

一、先看这条约定到底做了什么

两位判读者看同一批病例,各带一口自己的零点偏移:资深者 A 读出「真值 + a」,新手 B 读出「真值 + b」。两人对同一病例的差就是 b − a。

「以资深者为准」的做法是:把 B 整体减去这口差。代进去一步就出来:

B − (b − a) = 真值 + a

也就是说,校正之后剩下的偏差,精确等于资深者自己的偏倚 a。一分不多,一分不少。这与本次方向判断对错毫无关系——方向判断对,残差是 a;判断反了,残差变成 2b − a。两者都不是「错了一次」这么轻的说法,前者是「白校正」,后者是「把偏倚从 b 放大到 2b − a」。

二、「方向错误率」这个量在数学上没有位置

设 b = 5 pp。a 从 0 到 4 pp 逐个代进去,衡量「校正后误差平方」与「不校正误差平方」的比值:0.000、0.040、0.250、0.640。

到 a = 5 pp(也就是 a 与 b 一样大)时,这个比值恰为 1.000。这意味着校正跑了整整一套流程,结果和不校正一模一样,连百分之一的改善都没有。再往上,a = 7.5 pp 时比值 2.25,a = 10 pp 时 4.00。

这条曲线的形状说明:真正的开关不是一个概率,而是 a 与 b 的比值。它小于 1 就赚,等于 1 就白干,大于 1 就赔。而「错一次翻倍」这件事,只在 a = 2b 这个特殊点上才成立——那时比值恰好是 4。

三、抽样噪声造成的方向错误率,小到可以不谈

有人会说:上面假定 b − a 估得准,可它是估出来的,估歪了方向不就反了?这个担心在数值上站不住。

两人的差值均值,其标准误是根号二乘单次噪声除根号 n。单次噪声取 0.5 pp、n = 120 时,标准误只有 0.0645 pp。真差 5 pp 对应的 z 值是 77 个标准差——方向判错的概率小于十的负十三次方。这不是「很少错」,是「错不了」。

要让它开始错,真差得小到什么程度?反解出来:错误率降到 15% 需要真差小于 0.067 pp;降到 30% 需要小于 0.034 pp。而这个量级的差,本来就落在「校不校正都无所谓」的区间里。实测蒙特卡洛:真差 0.1 pp 时错误率 6.12%(解析值 6.07%),真差 0.05 pp 时 21.9%。

换句话说,backlog 里设想的那三档——5%、20% 至 30%、接近 50%——没有一档能由抽样噪声产生。

四、真正会出错的是换场景,错误率等于 2p(1−p)

第二个来源才有意义:在 A 场景标定,拿到 B 场景去用。各场景的真实偏倚差 δ 服从均值 μ、标准差 τ 的分布。两个场景方向相反的概率是

2Φ(μ/τ) × [1 − Φ(μ/τ)]

代入:μ = 5 pp、τ = 5 pp,错误率 26.7%;τ = 3.6 pp,15.1%;τ = 3.1 pp,10.1%;τ = 2 pp,1.2%。要压到 15% 以下,τ 必须不超过 3.59 pp;压到 10%,不超过 3.09 pp。

τ 本身估得出来吗?可以。15 份研究的条件下,真 τ = 4 pp 时它的 95% 置信区间半宽约 1.48 pp。所以「错误率 15%」这条线是可检验的,但它检验的对象是跨场景异质性,不是「这条约定本身可不可信」。

五、两种失败必须分开,混在一起判据就乱了

第一种是系统性错:资深者自己就比新手偏得更多(a 大于 b)。这时每一次的残差都大于 b,误差平方比是比值的平方。比值 1.5 得 2.25,比值 2 得 4.00。这不是「多少比例会错」,是百分之百都错,而且平均不掉——固定同一位资深者,算 1 份、5 份、15 份、50 份研究,残差恒为 1.000、1.000、1.000、1.000 pp。

第二种是随机错:每份研究换一对读者,a 围绕 2.5 pp 波动、标准差 2.5 pp。这时平均才有效:1 份研究残差 3.54 pp,5 份 2.74 pp,15 份 2.58 pp,50 份 2.53 pp。压掉的只是随机那部分,那 2.5 pp 的系统项一直留在那里。

把这两种当成同一个「随机错误率」,就会得出「错 20% 还能接受」的结论——而那正是系统性错时最危险的说法。

六、外部锚的自指困境

要判比值 a/b,就得知道 a。而知道 a 需要一个比两人都准的外部尺——那恰恰是校正想省掉的东西。

量化一下:外部量的偏倚取 1 pp(b = 5 pp),比值估计就偏 0.20;取 3 pp,偏 0.60;取 5 pp,偏 1.00。也就是说,只要外部量的偏倚达到新手的一半以上,「比值小于 0.9 可以校正」这个判断本身就判不出来了。

这不是悲观,是边界。这条约定不能被验证,只能被限定。

七、可以执行的报法

三项必须同时写出来:① b 的点估计与置信区间;② 比值的上界,由外部量精度反推;③ 是固定一位资深者,还是每份研究换人。

判据:上界小于 0.9,可用;跨在 0.9 至 1.1 之间,判定为打平,直接报未校正值,别把「校正后」当成果;大于 1.1,禁用,因为它不是偶尔错,是每次都错。

文献里正反两面都有支撑。正面:用多面 Rasch 模型估出并移除评分者效应后,分数与外部变量的相关性确实提高(PMID 36591072);同一量表跨自评与医生评的可比性检验也走这条路,且发现自评系统性高于医生、但幅度小(PMID 25546697)。反面:简单比值型校正会给终身关联高估 29%,原因是当前水平并非唯一驱动(PMID 16051613)——这正是比值大于 1 的真实例子。而方法比较本身,原作者早就说过应当报差值而不是报相关(PMID 22129533)。

证据与局限

已验证:校正后残差恒等于资深者自身偏倚,是代数恒等式,不需要数据支撑;误差平方比等于两人偏倚比的平方(多组参数核到小数点后四位)。

部分验证:抽样噪声导致的方向错误率极小(n ≥ 60 时低于十的负十三次方),已用解析式与四十万次蒙特卡洛互校一致。

待验证:跨场景异质性 τ 在真实判读领域的取值完全没测过,本文所有 26.7%、15.1%、10.1% 都建在假设的 μ 与 τ 上;比值的上界如何由外部量精度反推,也还没做过实测标定。

相关阅读

证据等级: C
参考文献: PMID 22129533, PMID 25546697, PMID 42306452, PMID 36591072, PMID 16051613, PMID 37336529, PMID 33853549, PMID 30383398

L1 网络状态:EDGE_TABLE 第 202 行「E-060 标签前提」——「KB-292(conflict):中医医师间诊断一致性在开放人群中『低(low)』,作者称为『令人担忧』(PMID 30383398);同系列第二篇指出偶然校正一致性统计常被误用、建议 Gwet AC2;但训练可改善(PMID 19538100)」——状态标为「冲突(未裁决)」。本文按冲突双方并列处理:一致性低是实测(30383398),训练可改善也是实测(19538100),两者并不互斥,因为前者测的是开放人群的原始一致性、后者测的是训练后的变化量。本文的恒等式只在「两人各自带一口恒定偏移」这个模型下成立,若偏移本身随病例亚型变化(PMID 42306452 的差异性评分者功能正是此情形),残差就不再是一个常数。

证据等级说明:A 表示有 PubMed 直接支撑该结论,B 表示科学类比或间接证据,C 表示传统文化共识或初步探索。本文标 C,是因为该边在 L1 内核中处于冲突未裁决状态,且本文核心结论是代数推导而非实测。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Based on the Chinese text provided, here is the translation of the title into English:

Evidence level: C

A single identity equation invalidates this question: shifting the novice onto the senior expert leaves behind only the senior expert’s own bias—it has nothing to do with “whether this directional judgment was right or wrong,” nor with “how often they err.”

I. What This Convention Actually Does

Two raters assess the same set of cases, each carrying their own zero-point offset: Senior Expert A reads “True Value + a”, Novice B reads “True Value + b”. The difference between them on the same case is b − a.

The “rely on the senior expert” approach: subtract this entire difference from B. Substituting once yields:

B − (b − a) = True Value + a

Thus, after correction, the remaining bias equals exactly the senior expert’s own bias a. Not one bit more, not one bit less. This is entirely unrelated to whether the directional judgment was correct—if the direction is right, the residual is a; if the direction is wrong, the residual becomes 2b − a. Neither represents “an error once” so lightly; the former is “white correction,” the latter “amplifying the bias from b to 2b − a.”

II. “Direction Error Rate” Has No Mathematical Standing

Set b = 5 pp. Substituting a from 0 to 4 pp stepwise, measuring the ratio of “corrected error squared” to “uncorrected error squared”: 0.000, 0.040, 0.250, 0.640.

At a = 5 pp (i.e., a equals b), this ratio exactly equals 1.000. This means the correction runs through the full procedure yet yields results identical to no correction, with not even a one-hundredth improvement. Beyond that, at a = 7.5 pp the ratio is 2.25, and at a = 10 pp it is 4.00.

The shape of this curve indicates: the true switch is not a probability, but the ratio of a to b. Below 1 it gains, equal to 1 it is wasted effort, above 1 it loses. And “doubling the error once” holds only at the special point a = 2b—where the ratio is exactly 4.

III. Sampling Noise Makes Direction Error Rate Negligible

One might argue: above we assumed b − a is estimated accurately, but since it’s estimated, if the estimate is off, the direction flips? This concern fails numerically.

The standard error of the mean difference between two raters is √2 times single-rater noise divided by √n. With single-rater noise at 0.5 pp and n = 120, the standard error is only 0.0645 pp. A true difference of 5 pp corresponds to a z-score of 77 standard deviations—probability of directional error less than 10⁻¹³. This is not “rarely wrong,” it is “cannot be wrong.”

To make error begin, how small must the true difference be? Solving backwards: error rate drops to 15% only if true difference < 0.067 pp; to 30% only if < 0.034 pp. Such magnitudes fall squarely into the “correct or not, doesn’t matter” zone. Monte Carlo simulation confirms: true difference 0.1 pp yields error rate 6.12% (analytical value 6.07%); true difference 0.05 pp yields 21.9%.

In short, none of the three backlog tiers—5%, 20–30%, near 50%—can arise from sampling noise.

IV. Real Errors Come from Cross-Scenario Shifts; Error Rate Equals 2p(1−p)

The second source matters: calibrate in Scenario A, apply to Scenario B. True bias differences δ across scenarios follow a distribution with mean μ and standard deviation τ. Probability the two scenarios have opposite direction is

2Φ(μ/τ) × [1 − Φ(μ/τ)]

Substituting: μ = 5 pp, τ = 5 pp → error rate 26.7%; τ = 3.6 pp → 15.1%; τ = 3.1 pp → 10.1%; τ = 2 pp → 1.2%. To keep error below 15%, τ must not exceed 3.59 pp; to keep below 10%, τ ≤ 3.09 pp.

Can τ be estimated? Yes. Under 15 studies, with true τ = 4 pp, its 95% confidence interval half-width is about 1.48 pp. Thus “error rate 15%” is testable, but it tests cross-scenario heterogeneity, not “whether this convention itself is trustworthy.”

V. Two Failures Must Be Separated; Mixing Them Confuses Criteria

First, systematic error: the senior expert is inherently more biased than the novice (a > b). Then every residual exceeds b, and the error-squared ratio is the square of the ratio. Ratio 1.5 gives 2.25, ratio 2 gives 4.00. This is not “what proportion will err,” it is 100% always wrong, and never averages down—fixing the same senior expert, computing 1, 5, 15, or 50 studies, the residual stays constant at 1.000, 1.000, 1.000, 1.000 pp.

Second, random error: each study pairs different raters, with a fluctuating around 2.5 pp and standard deviation 2.5 pp. Then averaging matters: 1 study residual 3.54 pp, 5 studies 2.74 pp, 15 studies 2.58 pp, 50 studies 2.53 pp. Only the random part gets suppressed; the 2.5 pp systematic term remains.

Treating both as one “random error rate” leads to the conclusion “20% error is acceptable”—which is precisely the most dangerous claim under systematic error.

VI. The Self-Referential Dilemma of External Anchors

To judge the ratio a/b, we must know a. Knowing a requires an external scale more accurate than either—exactly what correction aims to avoid.

Quantify: external bias taken as 1 pp (b = 5 pp), ratio estimate biased by 0.20; taken as 3 pp, biased by 0.60; taken as 5 pp, biased by 1.00. Thus, once external bias exceeds half the novice’s, “ratio < 0.9 allows correction” cannot even be judged.

This is not pessimism, it’s a boundary. This convention cannot be validated, only bounded.

VII. Executable Reporting Format

Three items must be reported simultaneously: ① point estimate and confidence interval of b; ② upper bound of the ratio, inferred from external scale precision; ③ whether one fixed senior expert or a new person per study.

Criteria: upper bound < 0.9 → usable; spanning 0.9 to 1.1 → call it a tie, report the uncorrected value directly, do not treat “after correction” as achievement; > 1.1 → forbidden, because it never errs randomly—it always errs.

Literature supports both sides. Positive: after estimating and removing rater effects using multi-faceted Rasch models, correlation with external variables indeed improves (PMID 36591072); cross-comparability of self-ratings vs physician ratings also follows this path, finding self-rating systematically higher but smaller magnitude (PMID 25546697). Negative: simple ratio-based correction overestimates lifetime association by 29%, because current level is not the sole driver (PMID 16051613)—this is exactly the ratio > 1 case. Moreover, method comparison itself, authors long ago stated should report difference rather than correlation (PMID 22129533).

Evidence and Limitations

Verified: After correction, residual equals the senior expert’s own bias—a purely algebraic identity, requiring no data; error-squared ratio equals the square of the bias ratio (multi-parameter kernels to four decimal places).

Partially Verified: Sampling noise-induced direction error rate is negligible (n ≥ 60, below 10⁻¹³), confirmed by analytical formula and 400,000 Monte Carlo runs.

To Be Verified: Cross-scenario heterogeneity τ has never been measured in real diagnostic domains; all our 26.7%, 15.1%, 10.1% rest on assumed μ and τ; how the ratio’s upper bound is inferred from external scale precision also lacks empirical calibration.

Related Reading

Evidence Level: C
References: PMID 22129533, PMID 25546697, PMID 42306452, PMID 36591072, PMID 16051613, PMID 37336529, PMID 33853549, PMID 30383398

L1 Network Status: EDGE_TABLE row 202 “E-060 label premise”—”KB-292 (conflict): diagnostic consistency among TCM physicians in open populations is ‘low (low)’, authors call it ‘concerning’ (PMID 30383398); the second paper in the series notes that chance-corrected agreement statistics are often misused, recommends Gwet AC2; yet training can improve (PMID 19538100)”—status marked “Conflict (undecided)”. This paper treats both sides of the conflict equally: low consistency is empirically measured (30383398), training can improve it is also empirically measured (19538100); they do not contradict because the former measures raw consistency in open populations, the latter measures change after training. Our identity equation holds only under the model “two raters each carry a constant offset”; if the offset itself varies by case subtype (PMID 42306452’s differential rater function is exactly this scenario), the residual is no longer constant.

Evidence Level Explanation: A indicates direct PubMed support for this conclusion; B indicates scientific analogy or indirect evidence; C indicates traditional consensus or preliminary exploration. This paper is marked C because this edge lies in a conflicted, undecided state within the L1 core, and our core conclusion is algebraic derivation rather than empirical measurement.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *