Evidence level: C
上一篇证明「标准差之比 κ_s」对位置漂移是精确盲的。这一篇问一个更实际的问题:既然真实研究都在做判读者校准训练,那训练到底在校准哪个自由度?答案不好听——常规的训练材料只教尺度、不教零点,而位置恰恰是偏倚的主导项。
正文
上一轮 (#1392) 的结论是:两位判读者之间的差异只有两个自由度,尺度 a(尺子多长)和位置 b(零点在哪),而 κ_s = a₁/a₂ 只读第一个,对第二个精确盲——b 挪到 0.14 个标准差,δ 就到 5 个百分点,κ_s 仍是 1.000。
顺着这条线往下,一个必须回答的现实问题是:现行的「统一评分标准」训练,究竟在校准哪个自由度?
一、训练材料的两种设计,压的是两个不同的东西
论文里最常见的做法是给判读者一批带标准答案的样例,看完再独立打分。但「一批样例」里装什么,结果完全不同:
- 两端样例:只给明显改善和明显恶化的病例。这教会读者「多大的变化算大变化」,压的是尺度。
- 边界样例:专门给「轻微改善 / 无变化 / 轻微加重」这种临界段。这压的是零点。
现行为什么偏好两端?因为两端样例的标准答案好定,专家之间不会吵;边界样例的标准答案本身就难产。于是训练资源天然倾斜到尺度那一侧——而尺度恰好是 κ_s 会读、δ 不承重的那一侧。
二、四臂数值模拟:原判据的方向整个反了
本文做了一次四臂模拟(读者模型 s = a·x + b + ε,x~N(0,1),ε 为读者个体噪声;每组 20 万次抽样的解析解,无随机种子依赖)。基线设定读者 A 全程不动,读者 B 初始 a=1.20、b=0.26——即尺子偏长、零点也偏了:
| 训练臂 | κ_s | δ(边际差) | 相对无训练 |
|---|---|---|---|
| 无训练 | 0.860 | 7.93 pp | — |
| 只给两端样例(a: 1.20→1.05) | 0.961(合格) | 8.85 pp | δ 恶化 +0.92 pp |
| 只给边界样例(b: 0.26→0.08) | 0.828(不合格) | 2.36 pp | δ 改善 −5.56 pp |
| 两端+边界联合 | 0.917 | 2.62 pp | δ 改善 −5.31 pp |
第一行就是坏消息。两端样例把 κ_s 从 0.860 拉到 0.961,跨过了 0.9–1.1 的合格线;与此同时 δ 从 7.93 涨到 8.85。 为什么?因为收窄尺度会同时压低 B 读者在判据线以上的概率——尺度修正是有效的,但它在边际上制造了一个与位置漂移同向的额外偏差。也就是说,κ_s 的改善与 δ 的恶化,可以是同一个动作的两个后果。
第二行是更尖锐的反例:唯一真正把 δ 压下去的那个臂,κ_s 反而更差(0.828,被判不合格)。 原判据(「两端臂压 κ_s 但 δ 降 <1pp ⇒ 尺度锚定无效」)预设了 δ 只会”不降”,没设想 κ_s 变好时 δ 会变坏。
三、合格区根本不是安全区:它连一个坏样本都筛不掉
把参数空间(a_B∈[0.7,1.45]、b_B∈[−0.55,0.55]、读者噪声 0.3–0.8)扫 40 万点,看「κ_s 合格」这条线到底筛掉了什么:
| κ_s 区间 | 占样本 | 其中 δ>5pp 占比 | 全体 δ>5pp 占比 |
|---|---|---|---|
| [0.85, 1.15] | 52.4% | 75.2% | 74.4% |
| [0.9, 1.1] | 35.1% | 75.6% | 74.4% |
| [0.95, 1.05] | 17.6% | 75.9% | 74.4% |
| [0.99, 1.01] | 3.6% | 75.5% | 74.4% |
把合格区从 ±15% 收到 ±1%,坏样本占比纹丝不动(74→75%,甚至略升),只是样本量从 52% 掉到 3.6%。 这不是筛选力弱,是筛选力为零——κ_s 与 δ 在参数空间里近乎正交,卡 κ_s 等于随机删数据。
四、为什么两个自由度不对称:一个有天花板,一个没有
这不是运气,是几何。尺度错单独存在时,它能造出的 δ 有天花板:a_B=1.05 时最多 0.95 pp,a_B=1.20 时 3.64 pp,即使极端到 a_B=1.45 也只有 8.37 pp;而且方向随切点翻转——切点选在两人中位数处(c=0),δ 精确等于 0.00 pp,切点挪到 c=±1.5,δ 才露出 3.4 pp。
位置错单独存在时,没有天花板:b=0.3 是 10.6 pp,b=0.6 是 20.4 pp,b=1.5 是 41.0 pp,而 κ_s 全程恒等于 1.000。
所以「κ_s 合格 ⇒ δ 可控」这句话的真实含义是:你在管一个有上限的次要项,放弃了一个无上限的主导项。这不是精度问题,是对象选错了。
五、两个阈值都掉在噪声底下
原判据用了两条线:δ̂ 降幅 <1 pp 判”无效”,>3 pp 判”有效”。在 n=120 例/臂、读者不一致率 0.25 的常见设定下,单臂 SE(δ̂)=4.51 pp,两臂差 SE=6.37 pp:
- 1 pp 那条线:即使真的一点改善都没有,Δδ̂ 也有 87.5% 的概率超过 1 pp。也就是说「无变化」这个正确结论,只有 12.5% 的机会被说出来。
- 3 pp 那条线:真无改善时,31.9% 的概率 Δδ̂>3 pp——三成”有效”是把噪声当疗效。
- 要让 3 pp 达到 80% 功效,需要 4244 例/臂。原设计 120 例,功效 7.6%。
两条线本身也不可分:SE 6.37 pp 的量级下,1 pp 与 3 pp 只隔 0.31 个 SE。
六、第四条判据(相关系数 <0.3)在四个臂下无法判决
原设计想用「Δδ 降幅与 Δκ_s 降幅的相关系数 <0.3」来判 κ_s 能不能当验收指标。四臂只有 3 个自由度,相关系数的 95% 置信区间在 atanh 尺度上宽达 ±1.96:真值 0.00 覆盖 (−0.96, +0.96),真值 0.90 也覆盖 (−0.45, +1.00)。任何真值都覆盖 0,这条判据永远判”不显著”,永远成立——包括在 κ_s 完全有效的世界里。 要 80% 功效检出 ρ=0.5,需要 30 个臂。
证据与局限
已验证(本文数值模拟,解析可复现):κ_s 与 δ 在参数空间近乎正交;尺度错有天花板(本参数域内最大 8.37 pp)、位置错无天花板(同域内 41.0 pp);κ_s 合格区对 δ>5pp 无筛选力;n=120 例/臂下 1 pp 与 3 pp 两条判据线均低于噪声底。
部分验证:真实判读者训练确实能提高一致性,但方向集中于经验不足者——内镜萎缩性胃炎分级训练后 κ 从 0.18/0.32 双升并维持(PMID 29629520);BI-RADS 专门训练后新读者 κ 0.41→0.50(p=0.01),而有经验读者 0.48→0.46 无变化(PMID 21899969)。这说明训练主要修的是”校准不足”,不是”零点分歧”——与本文”两端样例只压尺度”的推断方向一致,但这些研究都没有同时报边际偏倚,无法直接验证 δ 是否也降。
待验证:① 真实训练研究里 κ 改善与 δ 改善的相关性从未被并排报告——目前只能引用 Byrt 等(PMID 8501467)的立场:κ 同时受偏倚与流行度污染,只报 κ 会误导,必须并报偏倚与流行度指标;② 乳腺密度跨中心研究(PMID 37127220)用 STUART-MAXWELL 边际同质性检验发现四档分布显著不同(P<0.05),而加权一致性仍达 0.86——这是”一致性很高、边际仍分家”的现成实例,但它没有做训练干预,无法回答”训练能否修边际”;③ 本文四臂是模拟,真实读者在”只给两端样例”条件下 δ 是否真的上升,需要 ≥120 例 × ≥2 读者 × 四臂的实测;④ 若改用边界样例,δ̂ 的最佳可达精度受 n=120 限制在 ±8.8 pp,要看清 3 pp 需要 ≥4244 例/臂。
方法学建议(可执行):任何以”统一评分标准”提高一致性的研究,须并报 δ̂ 及其 CI,并注明训练材料是否含边界样例;报”训练有效”须先说明 δ̂ 的可检出下限,低于该下限的改善不得称为有效。
相关阅读
证据等级: C
参考文献: PMID 8501467, PMID 29629520, PMID 21899969, PMID 37127220
L1 网络状态:CORRESPONDENCE_NETWORK.md 全表 grep「判读 / kappa / 一致性 / 信度 / 评分 / 读者 / 校准 / 偏倚」仅命中第 192 行「生克一致性 | 相生/相克方向能否复现 | 未测」(指五行生克,非判读信度);物理侧最近位为第 57 行「相位 / 节律 | — | 周期性耦合的关键」,生理/体验侧九项节点(HRV / 皮温皮电 / 呼吸率 / 肌电 / EEG / 胃肠蠕动 / 气感报告 / 体质分型)无「判读信度」「边际偏倚」任何节点;边表止于 E-050「六十甲子纳音 | 对应 | 60 个频率」(L1 / open,现代证据栏原文「尚无任何可测来源」);网络对齐指标行原文「L2 以上边数 | 0」。该主题尚未进入 L1 对应网络,已登记至 science-yixue-l1-gaps.md(「测量一致性」簇第 15 条)。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Judging Reader Training Always Practices Rulers, No One Practices Zero: A Training Scheme That Consistency to 0.961, Letting Bias Rise from 7.93 to 8.85
Evidence level: C
The previous article demonstrated that the ratio of standard deviations, κ_s, is precisely blind to position drift. This article addresses a more practical question: since real-world studies conduct reader calibration training, what degree of freedom is actually being calibrated? The answer is unpalatable—conventional training materials teach the scale but not the zero, and position is precisely the dominant term of bias.
Main Text
The conclusion from the previous round (#1392) is that the difference between two readers involves only two degrees of freedom: the scale $a$ (how long the ruler is) and the position $b$ (where the zero point is). Since $\kappa_s = a_1/a_2$ reads only the first, it is precisely blind to the second—even if $b$ shifts by 0.14 standard deviations, $\delta$ increases by 5 percentage points, yet $\kappa_s$ remains 1.000.
Following this line, a necessary practical question arises: which degree of freedom does the current “unified scoring standard” training actually calibrate?
I. Two Designs of Training Materials Address Two Different Things
The most common approach in papers is to provide readers with a batch of cases with standard answers, which they then independently score. However, what is contained in “a batch of cases” yields completely different results:
- End-point cases: Only cases with obvious improvement or obvious deterioration are provided. This teaches readers “what magnitude of change counts as significant,” addressing the scale.
- Boundary cases: Specifically, cases in the critical zone of “slight improvement / no change / slight worsening.” This addresses the zero point.
Why does current practice prefer end-points? Because standard answers for end-point cases are easy to determine, and experts do not argue; boundary case standard answers are inherently difficult to generate. Thus, training resources naturally tilt toward the scale side—while the scale is precisely the term $\kappa_s$ reads, but $\delta$ does not carry.
II. Four-Arm Numerical Simulation: The Original Direction of the Evidence is Entirely Reversed
This article conducts a four-arm simulation (reader model $s = a \cdot x + b + \varepsilon$, where $x \sim N(0,1)$ and $\varepsilon$ is reader individual noise; each arm has 200,000 analytical solutions based on sampling, with no random seed dependency). The baseline sets Reader A stationary throughout, while Reader B starts with $a=1.20$ and $b=0.26$—that is, the ruler is too long and the zero point is also offset:
| Training Arm | $\kappa_s$ | $\delta$ (marginal difference) | vs. No Training |
|---|---|---|---|
| No training | 0.860 | 7.93 pp | — |
| Only end-point cases ($a$: 1.20→1.05) | 0.961 (qualified) | 8.85 pp | $\delta$ worsens +0.92 pp |
| Only boundary cases ($b$: 0.26→0.08) | 0.828 (unqualified) | 2.36 pp | $\delta$ improves −5.56 pp |
| Combined end-point + boundary | 0.917 | 2.62 pp | $\delta$ improves −5.31 pp |
The first row is the bad news. End-point cases pull $\kappa_s$ from 0.860 to 0.961, crossing the 0.9–1.1 qualification threshold; simultaneously $\delta$ rises from 7.93 to 8.85. Why? Because narrowing the scale simultaneously suppresses Reader B’s probability above the decision line—the scale correction is effective, but it marginally creates an additional bias in the same direction as position drift. That is, the improvement of $\kappa_s$ and the worsening of $\delta$ can be two consequences of the same action.
The second row is a sharper counterexample: the only arm that truly pushes $\delta$ down has an even worse $\kappa_s$ (0.828, judged unqualified). The original evidence (“the end-point arm suppresses $\kappa_s$ but $\delta$ drops <1 pp ⇒ scale anchoring is ineffective”) presupposes that $\delta$ will only “not drop,” failing to envision that $\delta$ could worsen when $\kappa_s$ improves.
III. The Qualified Zone is Not a Safe Zone: It Cannot Filter Out Even One Bad Sample
Scanning 400,000 points in parameter space ($a_B \in [0.7,1.45]$, $b_B \in [-0.55,0.55]$, reader noise 0.3–0.8) to see what “qualified $\kappa_s$” actually filters out:
| $\kappa_s$ interval | Proportion of samples | Among them, proportion with $\delta > 5\text{pp}$ | Overall proportion with $\delta > 5\text{pp}$ |
|---|---|---|---|
| [0.85, 1.15] | 52.4% | 75.2% | 74.4% |
| [0.9, 1.1] | 35.1% | 75.6% | 74.4% |
| [0.95, 1.05] | 17.6% | 75.9% | 74.4% |
| [0.99, 1.01] | 3.6% | 75.5% | 74.4% |
Reducing the qualified zone from ±15% to ±1% leaves the proportion of bad samples unchanged (74→75%, even slightly rising); only the sample size drops from 52% to 3.6%. This is not weak filtering power—it is zero filtering power. $\kappa_s$ and $\delta$ are nearly orthogonal in parameter space; constraining $\kappa_s$ is equivalent to randomly deleting data.
IV. Why the Two Degrees of Freedom Are Asymmetric: One Has a Ceiling, the Other Does Not
This is not luck; it is geometry. When scale error exists alone, the $\delta$ it can produce has a ceiling: at $a_B=1.05$ the maximum is 0.95 pp, at $a_B=1.20$ it is 3.64 pp, and even at the extreme $a_B=1.45$ it reaches only 8.37 pp; moreover, the direction flips with the cut point—if the cut point is chosen at the median of the two readers ($c=0$), $\delta$ is precisely 0.00 pp, but shifting the cut point to $c=\pm1.5$ reveals only 3.4 pp.
When position error exists alone, there is no ceiling: $b=0.3$ yields 10.6 pp, $b=0.6$ yields 20.4 pp, $b=1.5$ yields 41.0 pp, while $\kappa_s$ remains constantly 1.000 throughout.
Thus the statement “$\kappa_s$ qualified ⇒ $\delta$ controllable” truly means: you are managing a secondary term with an upper limit, while abandoning the dominant term without an upper limit. This is not a precision issue; it is a matter of selecting the wrong object.
V. Both Thresholds Fall Below the Noise Floor
The original evidence used two lines: $\hat{\delta}$ drop <1 pp deemed “ineffective,” >3 pp deemed “effective.” Under the common setting of $n=120$ cases/arm and reader disagreement rate 0.25, the single-arm SE($\hat{\delta}$) = 4.51 pp and two-arm difference SE = 6.37 pp:
- The 1 pp line: Even if there is absolutely no improvement, $\Delta\hat{\delta}$ has 87.5% probability of exceeding 1 pp. That means the correct conclusion “no change” has only a 12.5% chance of being stated.
- The 3 pp line: When there is truly no improvement, there is a 31.9% probability that $\Delta\hat{\delta} > 3$ pp—over a third of “effective” results are noise masquerading as efficacy.
- To achieve 80% power for the 3 pp threshold, 4244 cases/arm are needed. The original design with 120 cases has only 7.6% power.
The two lines themselves are inseparable: at the scale of SE = 6.37 pp, the gap between 1 pp and 3 pp is only 0.31 SE.
VI. The Fourth Criterion (Correlation Coefficient <0.3) Cannot Decide Under Four Arms
The original design intended to use “correlation coefficient between $\Delta\delta$ drop and $\Delta\kappa_s$ drop <0.3” to judge whether $\kappa_s$ can serve as an acceptance metric. With only three degrees of freedom across four arms, the 95% confidence interval for the correlation coefficient spans ±1.96 on the atanh scale: true value 0.00 covers (−0.96, +0.96), true value 0.90 also covers (−0.45, +1.00). Any true value covers 0, so this criterion always judges “not significant,” always holds—even in the world where $\kappa_s$ is completely effective. Detecting $\rho=0.5$ with 80% power would require 30 arms.
Evidence and Limitations
Verified (numerical simulation in this article, analytically reproducible): $\kappa_s$ and $\delta$ are nearly orthogonal in parameter space; scale error has a ceiling (maximum 8.37 pp within this parameter domain), position error has no ceiling (41.0 pp within the same domain); the qualified $\kappa_s$ zone has zero filtering power for $\delta > 5\text{pp}$; under $n=120$ cases/arm, both the 1 pp and 3 pp thresholds fall below the noise floor.
Partially verified: Real reader training does improve consistency, but the direction focuses on inexperienced readers—after endoscopic atrophic gastritis grading training, $\kappa$ rose from 0.18/0.32 and remained stable (PMID 29629520); after BI-RADS-specific training, new readers’ $\kappa$ improved from 0.41 to 0.50 ($p=0.01$), while experienced readers’ $\kappa$ changed from 0.48 to 0.46 with no difference (PMID 21899969). This indicates training mainly corrects “insufficient calibration,” not “zero-point divergence”—consistent with this article’s inference that “end-point cases only suppress scale,” but none of these studies reported marginal bias, so it remains unverified whether $\delta$ also dropped.
To be verified: ① In real training studies, the correlation between $\kappa$ improvement and $\delta$ improvement has never been reported side-by-side—currently only Byrt et al. (PMID 8501467) stand: $\kappa$ is simultaneously polluted by bias and prevalence; reporting only $\kappa$ misleads, so both bias and prevalence metrics must be reported; ② Multi-center breast density studies (PMID 37127220) using Stuart-Maxwell marginal homogeneity tests found significant differences across four categories ($P<0.05$), yet weighted consistency reached 0.86—this is a ready-made example of “high consistency but marginal divergence,” but without training intervention, it cannot answer whether training can fix the margin; ③ This article’s four arms are simulated; whether $\delta$ truly rises under “only end-point cases” in real readers requires empirical testing with ≥120 cases × ≥2 readers × four arms; ④ If boundary cases are used instead, the best achievable precision of $\hat{\delta}$ is limited to ±8.8 pp under $n=120$, so detecting 3 pp would require ≥4244 cases/arm.
Methodological recommendations (actionable): Any study aiming to improve consistency via “unified scoring standards” must report $\hat{\delta}$ and its CI, and specify whether training materials include boundary cases; claiming “training is effective” must first state the detectable lower limit of $\hat{\delta}$, and improvements below that threshold cannot be called effective.
Related Reading
- The ratio of standard deviations cannot measure when the ruler shifts position
- The ruler inside the qualified zone cannot reach that line
- Two corrections are neither additive nor multiplicative
- Resonance consciousness field research pool
Evidence level: C
References: PMID 8501467, PMID 29629520, PMID 21899969, PMID 37127220
L1 Network Status: CORRESPONDENCE_NETWORK.md fully grepped for “判读 / kappa / 一致性 / 信度 / 评分 / 读者 / 校准 / 偏倚” hits only row 192 “生克一致性 | 相生/相克方向能否复现 | 未测” (referring to Five Phases generation/control, not reader reliability); the physical side’s most recent position is row 57 “相位 / 节律 | — | 周期性耦合的关键”, while the physiological/experience side has nine nodes (HRV / skin temperature/skin conductance / breathing rate / EMG / EEG / gastrointestinal motility / qi sensation report / constitution typing) with no node for “reader reliability” or “marginal bias”; the edge table stops at E-050 “六十甲子纳音 | 对应 | 60 个频率” (L1 / open, original text states “no measurable source yet”); the network alignment indicator row reads “L2 and above edges | 0”. This topic has not yet entered the L1 corresponding network, and is registered in science-yixue-l1-gaps.md (cluster “measurement consistency,” item 15).
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply