Evidence level: C
二分类一致性里藏着一个代数锁:κ 随边际分家单调上升,所以「把偏倚练掉了」这件事在 κ 上必然显示为变差;而受一致率上界约束,这一整条通道最多只值 0.007。
一、被问错的问题
1420 的问题设定是:筛出 ≥20 项训练前后双时点的判读研究,逐项算 Δκ 与 Δδ,做秩相关,看两者「同向、无关还是反向」。这个设计隐含一个前提——Δκ 与 Δδ 是两个可以各自变号、因而相关系数落在 −1 到 +1 之间的量。
前提不成立。在二分类、两位读者、真值给定的场合,κ 是 δ 的偶函数,且在可行域内严格单调递增。δ 无论朝哪边动,κ 都跟着同向动;δ 变小,κ 一定变小。相关系数被代数关系锁在正区间,不存在「无关」这个选项。问它俩是否相关,等于问 2+2 是否等于 4。
真正该问的是量级:这个耦合能大到把「训练有效」读成「训练有害」吗。
二、κ 的机会空间里藏着一块 δ²/2
记同意的观测比例为 P_o,两位读者的阳性判读率 a 与 b,平均流行度 m=(a+b)/2,边际分家 δ 为 a 与 b 之差的绝对值。Cohen 的 P_e = ab+(1−a)(1−b) 按 m 与 δ 重写:
P_e(κ) = m² + (1−m)² − δ²/2
代入 a=m+δ/2、b=m−δ/2 即可验证,我在三组参数上核到 1e-16。而 Gwet 的 AC1 用的 P_e = 2m(1−m),只含 m、不含 δ。两式相减得恒等式:
1 − P_e(κ) = P_e(AC1) + δ²/2
即 κ 的「机会以上空间」里,有一块 δ²/2 是边际分家贡献的。δ 越大,可分配空间越大,同样的 P_o 换来的 κ 越高。这一块从来没人算过。
三、方向是反的
固定 P_o 与 m,对 δ 求偏导:∂κ/∂δ = δ(1−P_o)/Q²,Q=1−P_e。非负,δ=0 时归零。κ 随 δ 单调上升。 于是「训练把读者切点对齐、δ 从 20pp 修到 2pp」这件事,在 κ 上的表现是 κ 下降。
而 κ 对 P_o 的偏导是 1/Q,正且远大于前者。两者之比给出交换率:一个百分点的 P_o 提升,等价于 Q/(δ(1−P_o)) 个百分点的偏倚。P_o=0.85、δ=15pp 时是 22.3 : 1,δ=9pp 时 36.6 : 1。偏倚要动二十多个百分点,才抵得上一致率动一个百分点。
四、但这条通道在可行域里长不大
P_o 有上界。给定 m 与 δ,P_o 不超过 min(a,b)+min(1−a,1−b),这个上界恰等于 1−δ。因此 δ ≤ 1−P_o:P_o=0.85 的数据,δ 最多 15pp;P_o=0.90 最多 10pp。高一致率数据天然排斥大偏倚——不是生理事实,是算术事实。
代入第三节:δ 从 0 拉到可行上界,κ 最多涨多少?P_o=0.85、m=0.43 时从 0.6940 到 0.7009,+0.0069,相对 1.0%;P_o=0.90 时 +0.0021(0.3%)。只有 P_o 低到 0.70,上界才有 30pp、值 κ +0.0515(13.3%)。
这把 #1402 的模拟结果推进了墙角。它报的是 κ 从 0.860 升到 0.961 的同时 δ 从 7.93pp 涨到 8.85pp。κ=0.86 要求 P_o 在 0.9 以上,那里 δ 上界连 10pp 都不到;而 0.92pp 的 δ 涨幅,按 ∂κ/∂δ 只值 κ +0.0004 量级。0.101 的涨幅不可能来自 δ,只能来自 P_o 或 m。方向对了,量级差两个半数量级。
五、反推两篇真研究
PMID 29629520:12 位内镜医师,训练前后 κ 从 0.18 升到 0.32(经验组)。少见的是它还报了诊断率 43.1%±10.7 变成 46.8%±5.9。把读者间诊断率的标准差折算成 δ 的代理(两读者差值的期望约为 1.128 倍标准差),m 由诊断率给出 0.431 与 0.468,得 δ 从 12.1pp 降到 6.7pp。代入 P_e 公式:P_e 从 0.5022 到 0.4998,只降 0.24pp。
若 κ 真从 0.18 涨到 0.27,反解 P_o 须从 0.5918 升到 0.6349,涨 4.3pp。归因:偏倚通道贡献 κ +0.0048,一致率通道 +0.0865。96% 的涨幅来自一致率,4% 来自偏倚——而偏倚的方向是变好,它对 κ 的贡献是变差。
结论不是「训练在刷偏倚」,恰恰相反:训练压根没在动偏倚这一维,因为它动不了 κ。唯一推得动 κ 的是一致率。
PMID 21899969:55 位乳腺钼靶阅片者,训练后总体 κ 0.44 到 0.48(p=0.14),新读者 0.41 到 0.50(p=0.01),老读者 0.48 到 0.46(p=0.60,判无差异)。这个下降正是本轮的预测:设 P_o=0.87、m=0.40,δ 从 13pp 修到 5pp,κ 从 0.7339 落到 0.7299,−0.0040——落在 p=0.60 的不显著里,与「什么都没发生」不可区分。真实的改善被判成无效应,而虚假的提升被判成有效应。两项研究都没报边际,两个判读都无法复核。
这两条合起来,正好落在 L1 那条仍未裁决的冲突上。E-060 标签前提两侧各有一批证据:一侧是 PMID 30383398 在开放人群里测出中医师间诊断一致性「低」、作者称令人担忧;另一侧是同系列第二篇(PMID 31730403 / 31634000 / 31730404)指偶然校正统计常被误用、应改 Gwet AC2,加上 PMID 19538100 报训练能把读者间相关从 0.112 抬到 0.618。本轮不站任何一边,只指出这个冲突里有一层没被分开:「一致性低」与「训练能提高一致性」这两个观察,都可以在切点分歧完全没动的情况下由 P_o 变化造出来。 低 κ 不等于切点分歧大,高 κ 也不等于切点对齐了。要裁决它,两边都缺同一个数——逐读者边际。
六、能不能反解回来
能,但要三个数。由恒等式解出:
δ² = 2[ (1−P_o)/(1−κ) + (1−P_o)/(1−AC1) − 1 ]
五组参数上核到 1e-15 精确。代价是必须同时报 κ、AC1 与 P_o。
只报 κ 时,δ 的可辨识区间有多宽?P_o=0.85、κ=0.18 时 δ 落在 0 到 60.5pp 之间,宽度超过任何一个实际偏倚;κ=0.32 时宽 66pp。这不是噪声,是结构性不可辨识——一个方程解两个未知数。
还有一处陷阱:AC1−κ 这个「指纹」自己会归零。gap=0 的充要条件是 δ 等于 √2 倍的 (2m−1) 的绝对值。m=0.43 时 δ=19.8pp,m=0.45 时 14.1pp——正落在常见域内。所以它不能当无条件可用的诊断量,必须先报 m。另有 PMID 33853549 指出加权系数在中低一致性、大样本、类别数多时精度显著下降,恰是本领域的常态区间。
证据与局限
已验证:P_e 的 m–δ 分解与恒等式 1−P_e(κ)=P_e(AC1)+δ²/2 是纯代数,数值核到机器精度;δ 的可行上界等于 1−P_o 亦为代数结论;κ 对 δ 的单调性与偶函数性质由偏导直接给出。方法学侧有 PMID 8501467 早就在说「只报 κ 会误导、必须并报偏倚与流行度」,本轮只是把这句话算成了具体数字。
部分验证:两篇真研究的反解用的是「读者间标准差折算 δ」这个代理,该关系只在正态近似下成立,原文未报逐读者边际,故反解值是估计而非实测。96% 与 4% 的归因比例随之带不确定度。
待验证:本轮的「δ 通道最高只值 0.007」建在 P_o=0.85、m=0.43 这一组参数上,若真实研究普遍落在更低一致率区(P_o<0.75),该通道可放大到 0.03–0.05,结论须改口径。另 AC1−κ 指纹在 m 接近 0.43 处失效,替代指纹尚未找到。
L1 网络状态:E-060 标签前提 —「KB-292(conflict):中医医师间诊断一致性在开放人群中「低(low)」,作者称为「令人担忧」(PMID 30383398);同系列第二篇指出偶然校正一致性统计常被误用、建议 Gwet AC2(PMID 31730403 / 31634000 / 31730404);但训练可改善(PMID 19538100)、量表化可行」→「用五脏标签做分组」这一步本身带噪声 — 状态:冲突(未裁决)/ conflict。
相关阅读
证据等级: C
参考文献: PMID 8501467, PMID 29629520, PMID 21899969, PMID 30383398, PMID 19538100, PMID 25090041, PMID 37127220, PMID 33853549
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Training bias removed, yet acceptance metrics worsen: κ is an even function of marginal splits, yet its maximum value is only 0.007
Evidence level: C
In binary agreement metrics lies an algebraic lock: κ rises monotonically with marginal shift, so “removing bias” necessarily registers as worsening κ; constrained by the upper bound of agreement, this entire channel caps at 0.007.
I. The Question Asked Wrong
Question #1420 posits: screen for ≥20 studies comparing readings before and after training, compute Δκ and Δδ, perform rank correlation, and determine if they are “same-direction, unrelated, or opposite.” This design implicitly assumes Δκ and Δδ can independently flip signs, placing the correlation coefficient between −1 and +1.
This premise fails. In binary classification with two readers and a given truth, κ is an even function of δ (δ is the absolute difference between readers’ thresholds). Within the feasible domain, κ strictly increases monotonically with δ. Regardless of the direction of change in δ, κ moves in the same direction; if δ decreases, κ must decrease. The correlation coefficient is algebraically locked into the positive interval; “unrelated” is not an option. Asking whether they correlate is like asking if 2+2 equals 4.
The real question is magnitude: Can this coupling be large enough to make “training effective” read as “training harmful”?
II. A δ²/2 Hides in κ’s Opportunity Space
Let $P_o$ be the proportion of agreeing observations, $a$ and $b$ the positive reading rates of two readers, and average prevalence $m=(a+b)/2$. Marginal shift $\delta$ is the absolute difference between $a$ and $b$. Cohen’s $P_e = ab+(1-a)(1-b)$ rewritten in terms of $m$ and $\delta$ is:
$$P_e(\kappa) = m^2 + (1-m)^2 – \frac{\delta^2}{2}$$
Substituting $a=m+\delta/2$ and $b=m-\delta/2$ verifies this identity to machine precision (1e-16) across three parameter sets. Gwet’s AC1 uses $P_e = 2m(1-m)$, containing only $m$ and not $\delta$. Subtracting the two yields the identity:
$$1 – P_e(\kappa) = P_e(AC1) + \frac{\delta^2}{2}$$
Thus, within κ’s “opportunity space,” a block of $\delta^2/2$ is contributed by marginal shift. Larger $\delta$ means more allocatable space, yielding higher κ for the same $P_o$. This component has never been calculated.
III. The Direction is Reversed
Fixing $P_o$ and $m$, the partial derivative with respect to $\delta$ is:
$$\frac{\partial \kappa}{\partial \delta} = \frac{\delta(1-P_o)}{Q^2}, \quad Q=1-P_e$$
This is non-negative, zero only when $\delta=0$. κ rises monotonically with δ. Therefore, “training aligns reader cutpoints, reducing $\delta$ from 20pp to 2pp” manifests as κ decreasing.
The partial derivative of κ with respect to $P_o$ is $1/Q$, positive and far larger than the former. Their ratio gives the exchange rate: one percentage point increase in $P_o$ equals $Q/[\delta(1-P_o)]$ percentage points of bias shift. For $P_o=0.85, \delta=15pp$, the ratio is 22.3:1; for $\delta=9pp$, it is 36.6:1. Bias must shift by twenty-plus percentage points to match a one-percentage-point change in agreement.
IV. Yet This Channel Cannot Grow Large
$P_o$ has an upper bound. Given $m$ and $\delta$, $P_o$ cannot exceed $\min(a,b)+\min(1-a,1-b)$, which equals $1-\delta$. Thus $\delta \le 1-P_o$: with $P_o=0.85$, $\delta$ maxes at 15pp; with $P_o=0.90$, max is 10pp. High agreement data inherently rejects large bias—not physiological fact, but arithmetic fact.
Substituting into Section III: pulling $\delta$ from 0 to its feasible upper bound, how much can κ rise? For $P_o=0.85, m=0.43$, κ increases from 0.6940 to 0.7009, +0.0069 (1.0%); for $P_o=0.90$, +0.0021 (0.3%). Only if $P_o$ drops to 0.70 does the upper bound reach 30pp, yielding κ +0.0515 (13.3%).
This pushes #1402’s simulation results to the corner. It reports κ rising from 0.860 to 0.961 while $\delta$ grows from 7.93pp to 8.85pp. Since κ=0.86 requires $P_o > 0.9$, the upper bound for $\delta$ is under 10pp; yet the observed $\delta$ increase of 0.92pp, via $\partial \kappa / \partial \delta$, accounts for only κ +0.0004 magnitude. A 0.101 rise cannot come from δ; it must stem from $P_o$ or $m$. The direction is correct, but the magnitude differs by two and a half orders of magnitude.
V. Back-solving Two Real Studies
PMID 29629520: 12 endoscopists, κ rose from 0.18 to 0.32 (experience group). Rarely, it also reported diagnostic rate shifting from 43.1%±10.7 to 46.8%±5.9. Converting the standard deviation of inter-reader diagnostic rate into a proxy for $\delta$ (expected difference between two readers is ~1.128 times the SD), with $m$ given by diagnostic rates (0.431 and 0.468), $\delta$ drops from 12.1pp to 6.7pp. Plugging into the $P_e$ formula: $P_e$ falls from 0.5022 to 0.4998, only −0.24pp.
If κ truly rose from 0.18 to 0.27, back-solving $P_o$ requires it to jump from 0.5918 to 0.6349, a +4.3pp increase. Attribution: bias channel contributes κ +0.0048, agreement channel +0.0865. 96% of the rise comes from agreement, 4% from bias—and the bias direction is improvement, yet its contribution to κ is worsening.
Conclusion: “Training reduces bias” is incorrect; training did not move the bias dimension at all, because it cannot affect κ. Only agreement drives κ changes.
PMID 21899969: 55 mammography readers, post-training overall κ 0.44 to 0.48 (p=0.14), new readers 0.41 to 0.50 (p=0.01), old readers 0.48 to 0.46 (p=0.60, no difference). This decline matches our prediction: setting $P_o=0.87, m=0.40$, $\delta$ drops from 13pp to 5pp, κ falls from 0.7339 to 0.7299, −0.0040—falling within the p=0.60 non-significant range, indistinguishable from “nothing happened.” Real improvement was judged ineffective, while spurious increase was deemed effective. Both studies failed to report marginal shifts, making replication impossible.
Together, these land squarely on the L1 conflict yet undecided. The E-060 label premise has evidence on both sides: one side is PMID 30383398 measuring low inter-doctor diagnostic consistency in an open population, authors calling it “concerning”; the other side is the second paper of the series (PMID 31730403 / 31634000 / 31730404) noting chance-corrected statistics are often misused, recommending Gwet AC2, plus PMID 19538100 reporting training can raise inter-reader correlation from 0.112 to 0.618. This round takes no side, merely pointing out a layer in this conflict that remains unseparated: “Low consistency” and “Training improves consistency” can both arise from changes in $P_o$ even when cutpoint divergence remains unchanged. Low κ does not mean large cutpoint divergence, nor does high κ mean perfect alignment. To resolve it, both sides lack the same number: per-reader marginal shift.
VI. Can We Back-Solve?
Yes, but three numbers are required. Solving the identity yields:
$$\delta^2 = 2\left[ \frac{1-P_o}{1-\kappa} + \frac{1-P_o}{1-AC1} – 1 \right]$$
Verified to 1e-15 precision across five parameter sets. The cost is reporting κ, AC1, and $P_o$ simultaneously.
Reporting only κ: how wide is the identifiable range for $\delta$? With $P_o=0.85, \kappa=0.18$, $\delta$ spans 0 to 60.5pp, wider than any practical bias; with $\kappa=0.32$, width is 66pp. This is not noise, but structural unidentifiability—one equation solving two unknowns.
One trap: the “fingerprint” $AC1-\kappa$ itself vanishes. The necessary and sufficient condition for gap=0 is $\delta = \sqrt{2} \times |2m-1|$. For $m=0.43$, $\delta=19.8pp$; for $m=0.45$, $\delta=14.1pp$—both falling within common ranges. Thus it cannot serve as an unconditional diagnostic metric without reporting $m$. Additionally, PMID 33853549 notes weighted coefficients lose precision significantly in low-to-moderate agreement, large samples, and many categories—exactly the normal domain of this field.
Evidence and Limitations
Verified: The $m$–$\delta$ decomposition of $P_e$ and the identity $1-P_e(\kappa)=P_e(AC1)+\delta^2/2$ are purely algebraic, numerically verified to machine precision; the feasible upper bound of $\delta$ equals $1-P_o$, also an algebraic result; κ’s monotonicity and even-function nature with respect to $\delta$ follow directly from partial derivatives. Methodologically, PMID 8501467 long ago stated “reporting only κ misleads; bias and prevalence must also be reported”; this round simply computes those statements into concrete numbers.
Partially Verified: Back-solving the two real studies uses “inter-reader SD converted to $\delta$” as a proxy, valid only under normal approximation; since original papers did not report per-reader marginals, the back-solved values are estimates rather than measured. Consequently, the 96% and 4% attribution ratios carry uncertainty.
Pending Verification: This round’s claim that “the $\delta$ channel maxes at 0.007” rests on $P_o=0.85, m=0.43$. If real studies generally fall in lower agreement regions ($P_o<0.75$), this channel could expand to 0.03–0.05, requiring revision of conclusions. Furthermore, the $AC1-\kappa$ fingerprint fails when $m$ approaches 0.43; no alternative fingerprint has been found yet.
L1 Network Status: E-060 label premise — “KB-292 (conflict): inter-doctor diagnostic consistency in open population is ‘low’ (PMID 30383398), authors calling it ‘concerning’; the second paper of the series notes chance-corrected statistics are often misused, recommending Gwet AC2 (PMID 31730403 / 31634000 / 31730404); yet training can improve (PMID 19538100), scaling feasible” → “grouping by five-organs labels” step itself introduces noise — Status: Conflict (undecided) / conflict.
Further Reading
- Two Degrees of Freedom Severe Asymmetry: Position Drift Has No Ceiling, Scale Error Does
- What Is That “Pass Line” Actually: Shift of How Many Standard Deviations: Closed-Form Solution for Marginal Bias Band
- Weighted Consistency 0.86 and Significant Rejection of Marginal Homogeneity Coexist Why
- Resonant Consciousness Field: How Label Noise Pollutes Grouping
Evidence Level: C
References: PMID 8501467, PMID 29629520, PMID 21899969, PMID 30383398, PMID 19538100, PMID 25090041, PMID 37127220, PMID 33853549
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply