2.772 这个系数没错,错的是你以为它需要修正:相依折价就是样本量虚报

Written by

in

Language

Evidence level: C

上一篇担心「同一批受试者上的两个 kappa 不独立,2.772 会偏保守」。#1388 说这个修正能把门槛砍掉一半。实际算下来:修正量确实存在,但它不是一个新的统计量,而是「你的 n 被虚报了 1/(1−ρ) 倍」这件旧事的另一种写法——而真正高 ρ 的场合,恰好是真差为零、最不该给折扣的场合。

正文

一、先把「相依折价」翻译成人话

1385 给出可辨差下限:

D_min = B + 1.96√2 · SE

其中 1.96√2 = 2.772 来自「两条独立 95% 区间不重叠」。若两个 kappa 算在同一批受试者上、共享同一份抽样误差,则差值标准误应为 1.96√(2−2ρ)·SE。#1388 据此提出一条折扣表,并预告「ρ=0.5 时门槛降 41%」。

那条预告是错的,而且不止一处。但更值得先说清的是:整件事根本不需要新公式。

二、这个修正是一个恒等式,不是新发现

把 SE = σ/√n 代进去:

1.96√(2−2ρ)·σ/√n = 1.96√2 · σ/√n_eff,其中 n_eff = n/(1−ρ)

我用 5 组 ρ(0.15 / 0.30 / 0.50 / 0.70 / 0.85)逐点验算,两种算法的 D_min 差最大 5.55×10⁻¹⁷,其余四组差恰好 0。

结论:所谓「相依折价」在代数上完全等价于「你以为自己有 60 人,其实只有 60/(1−ρ) 人的独立信息」。 ρ=0.5 时 60 人只值 120 人的价——注意方向,n_eff 比 n 大,因为相依让两个 kappa 差得更近、差值的方差更小。

这一条改写把问题从「要不要修系数」变成「你的样本量报得虚不虚」,后者可以查、可以报、可以被审稿人一句话问穿。而前者只会变成一个没人核的修正因子。

顺带把 #1388 的算术也纠了:ρ=0.5 时 √(2−2ρ)/√2 = √0.5 = 0.707,SE 项降 29.3% 而不是 41%;再算上不折扣的 B 项,D_min 实际只降 28.0%(m=0.90、n=200 时从 0.337 到 0.243)。「近半要改判」这个说法需要 ρ>0.70 才站得住。

三、ρ 到底有多大:仿真给出的答案和原来的假设相反

1388 的理由是「共享同一批受试者的边际与个体难判程度,抽样误差高度正相关」。这个理由听着顺,但没测过。我用潜变量阈值模型做了分层仿真(三档、m=0.60/0.90、n=60/240、判读噪声 σ_e 0.35–0.75、读者风格偏移 σ_τ 0/0.25/0.5,每组 600 次重复):

共享结构 实测 ρ 范围
不同受试者 + 不同读者对(真独立) −0.079 ~ 0.123
同批受试者 + 换一组读者对 0.015 ~ 0.308
同批受试者 + 同一读者对 + 换场合 0.073 ~ 0.974

第二行是 #1388 描述的情形,ρ 中位数 0.111,上限 0.308,36 个格子里有 5 个实测为负值、10 个低于 0.05。也就是说:共享受试者几乎不产生相关——共享的只是「这批人难不难判」这一个共同成分,而它在 kappa 的总方差里占比很小。

真正把 ρ 推高的是共享读者阈值(同一对人马重复判读):σ_τ=0.25 时 ρ 到 0.5–0.86,σ_τ=0.5 时到 0.90+。ρ 的主来源是「读者风格固定」,不是「受试者同一批」。 #1388 把归因放错了地方,于是「ρ 可能大于 0.5」这个预判只在一种场景下成立:同一组判读者相隔一段时间再判一遍。

四、最危险的一格:折扣在最该保守的地方最狠

ρ 高意味着两个 kappa 高度同步——而它们最同步的时刻,是真实差值本来为零的时刻。上面第三行那个场景(同一读者对、换场合)真差恒为 0,我却实测到:

配置 实测 ρ D_min(不折扣) D_min(折扣后) 判「可辨」的假阳性率
m=0.90, n=60, σ_τ=0 0.256 0.273 0.238 2.10% → 4.40%
m=0.90, n=60, σ_τ=0.5 0.793 0.477 0.225 0.00% → 3.83%
m=0.60, n=60, σ_τ=0.5 0.908 0.607 0.205 0.00% → 2.37%

不折扣时假阳性率是 0–2.5%,套上 ρ 折扣后升到 2–4.4%。折扣把「同一批人在两个场合被同一组人判了两遍」这种最该判为无可比性的设计,放进了「可辨」的篮子。 这不是数值瑕疵:ρ 高的原因恰恰是两条读数出自同一条流水线,此时「差」这个量本身就没承载新信息。

所以折扣不能无条件套。它的适用前提是「两个 kappa 的比较本身有意义」,而 ρ→1 时这个前提自我取消(差恒为 0,任何门槛都失去对象)。ρ 应当同时被当作「折扣因子」和「可比性红牌」读。

五、折不掉的那部分:m 的惩罚

1388 最想解决的其实是这个:#1385 发现 m 越大越没资格比(m=0.90 时 D_min 0.337,m=0.60 时只有 0.143,差 2.36 倍)。折扣能补多少?

要让 m=0.90 的 D_min 降到 m=0.60 的水平,需要 ρ ≥ 0.84–0.86(n=200 时 0.842、n=60 时 0.859)。这个值只在「同一读者对 + 强风格偏移」的窄带里达到,而在最常见的「换一组读者对」场景(ρ≈0.11 中位数)下,降幅只有 5.4%。

即使把 ρ 拉到 0.5 并且真的想守住 0.05 那条线,n 仍需约 1084 人——ρ=0.5 只是把原来的 16,963 减到 1084,量级没变。

m 的惩罚不是相依性造的,所以相依性折不掉它。 这一点值得单独讲:m 大时 Σp²→1,分母 (1−Σp²)² 把 SE 炸开,这是 kappa 这个系数的定义域问题,不是抽样设计问题。想靠「反正我两次判读是同一批人」把它绕过去,方向就错了。

六、ρ 怎么估:三种结构三种估法

这一条比折扣表本身更有用。ρ 不能靠假设,得按共享结构分三种估法:

  • 换读者对、同受试者:ρ̂ 可以把同一批受试者随机劈成两半,两半各自算 kappa,用两半的相关系数当 ρ̂(半样本法),再按 Spearman-Brown 校正,虽然这一步本身又会引入新误差,但量级上够用。
  • 同读者对、换场合:直接拿两个场合的 kappa 算 Pearson 相关即可,这是最干净的一种,但要注意它有偏——两个场合共享的不只是读者风格,还有「这批片子本来就难」这个共同成分。
  • 完全独立样本:ρ 应为 0,报 0 即可,但若实测出显著的 ρ,说明两组样本有重叠或来自同一中心,那是数据管理问题,不是统计问题。

这三种估法给出的 ρ̂ 量级差半个数量级(换读者对中位数 0.11 vs 同读者对中位数 0.50),混用会让折扣表失去意义。所以报告规范该要求的不只是「报 ρ」,而是「报 ρ 且报它是在哪种共享结构下估的」。

七、该改的是什么

不是 2.772,是报告口径。任何跨样本比两个 kappa 的研究,把「n=60」换成「n_eff = 60/(1−ρ̂)」写出来,并同报三样:

  • ρ̂ 的估计方式(同一读者对重复判读 / 换读者对 / 独立样本)
  • 若 ρ̂>0.5,同时声明「两次判读共享读者与受试者,本比较的可比性本身存疑」
  • B、SE、D_min 三个数照旧(#1385 口径),D_min 用 √(2−2ρ̂)

证据与局限

已验证:ρ 折扣与 n_eff 折算的代数等价(5 组 ρ 逐点验算,最大偏差 5.55×10⁻¹⁷);ρ 的三层结构(独立 / 换读者对 / 同读者对)在 27 组参数网格上的分层结果;折扣后假阳性率升高(每组 3000 次重复);#1388 的 41% 降幅应为 29.3%(算术复核)。

部分验证:ρ 对 σ_τ 的单调依赖来自潜变量正态模型,真实判读者阈值分布未必正态,也未含「学习效应」(第二遍判得更一致,会把 ρ 推到比本文更高);三档设定固定,五档有序分类下 ρ 的绝对数值会变,但「共享读者主导、共享受试者边缘」这个定性结构应当保持。

待验证:① 真实三档裁决研究中 ρ 的实测分布从未被报过,本文给的是模型值不是实测值;② 本文的 D_min 沿用 #1385 的 B 闭式,而 B 本身在边际不等(#1384)与 TVD 偏离 δ(#1387)下要重算,ρ 与这两条修正的交互未算;③ 「ρ>0.5 即可比性存疑」这条红线的阈值是拍的,没有依据。

相关阅读

证据等级: C
参考文献: PMID 18430244, PMID 25321842, PMID 24488927, PMID 36123642
L1 网络状态:CORRESPONDENCE_NETWORK.md 三域节点表仅登记单点信号(物理侧「相位 / 节律 | — | 周期性耦合的关键」,语义为两振荡锁相;生理侧「呼吸率 / 呼吸模式 | 可穿戴」「气感报告 | 结构化自评」),全表检索「判读 / 信度 / 一致率 / 样本量 / 相依 / 有效样本量」全部零命中;边表止于 E-050(纳音↔频率,L1、open,现代证据栏原文「尚无任何可测来源」);网络对齐指标行原文「L2 以上边数 | 0」。该主题尚未进入 L1 对应网络。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

2.772 is correct; the error lies in assuming it requires adjustment: dependency discount equals inflated sample size.

Evidence level: C

In the previous post, concern was raised that two independent kappas calculated on the same batch of subjects would not be truly independent, potentially making 2.772 overly conservative. #1388 suggested this correction could halve the threshold. Actual calculation reveals: while a correction factor does exist, it is not a new statistic but merely another way of stating the old fact that “your reported n is inflated by 1/(1−ρ) times.” Moreover, in cases of truly high ρ, the true difference is zero—the exact scenario where discounting should be avoided.

Main Body

I. First, translate “dependency discounting” into plain English

1385 provides the lower bound for the discernible difference:

D_min = B + 1.96√2 · SE

where 1.96√2 = 2.772 originates from “two independent 95% intervals not overlapping.” If two kappas are calculated on the same batch of subjects, sharing the same sampling error, then the standard error of the difference should be 1.96√(2−2ρ)·SE. #1388 accordingly proposes a discount table and previews that “when ρ=0.5, the threshold drops by 41%.”

This preview is incorrect, and not just in one place. But more importantly, what needs to be clarified first is: the entire matter fundamentally does not require a new formula.

II. This correction is an identity, not a discovery

Substituting SE = σ/√n into the equation:

1.96√(2−2ρ)·σ/√n = 1.96√2 · σ/√n_eff, where n_eff = n/(1−ρ)

I verified this point-by-point using five sets of ρ (0.15 / 0.30 / 0.50 / 0.70 / 0.85). The maximum difference in D_min between the two methods is 5.55×10⁻¹⁷; for the other four sets, the difference is exactly zero.

Conclusion: so-called “dependency discounting” is algebraically completely equivalent to “you thought you had 60 people, but actually only have 60/(1−ρ) independent pieces of information.” When ρ=0.5, 60 people are worth the price of 120—note the direction: n_eff is larger than n because dependency makes the two kappa differences closer together, reducing the variance of the difference.

This reformulation shifts the problem from “whether to correct the coefficient” to “whether your reported sample size is inflated.” The latter can be checked, reported, and exposed by a single sentence from reviewers. The former merely becomes an unverified correction factor.

Incidentally, #1388’s arithmetic is also corrected: when ρ=0.5, √(2−2ρ)/√2 = √0.5 = 0.707, meaning the SE term drops by 29.3%, not 41%; including the undiscounted B term, D_min actually drops only by 28.0% (from 0.337 to 0.243 when m=0.90, n=200). The claim “nearly half must be reclassified” holds only when ρ>0.70.

III. How large is ρ? Simulation yields the opposite of the original assumption

1388’s reasoning is that “sharing the same batch of subjects makes marginal and individual difficulty hard to distinguish, leading to high positive correlation in sampling error.” This sounds plausible but was never tested. Using a latent variable threshold model, I performed stratified simulations (three tiers: m=0.60/0.90, n=60/240, reading noise σ_e 0.35–0.75, reader style shift σ_τ 0/0.25/0.5, with 600 repetitions per group):

Shared Structure Measured ρ Range
Different subjects + different reader pairs (truly independent) −0.079 ~ 0.123
Same batch of subjects + different reader pairs 0.015 ~ 0.308
Same batch of subjects + same reader pair + different occasions 0.073 ~ 0.974

The second row corresponds to #1388’s described scenario, where the median ρ is 0.111, the upper limit is 0.308, and among 36 grids, 5 had negative measured values and 10 were below 0.05. This means: sharing subjects almost does not produce correlation—the shared component is merely “how hard this batch of people is to judge,” which accounts for a very small portion of the total variance in kappa.

What truly drives ρ upward is shared reader thresholds (the same pair of readers repeatedly judging): when σ_τ=0.25, ρ reaches 0.5–0.86; when σ_τ=0.5, it exceeds 0.90. The main source of ρ is “fixed reader style,” not “same batch of subjects.” #1388 misattributed the cause, so the prediction “ρ may exceed 0.5” holds only in one scenario: when the same group of readers judges again after some time interval.

IV. The most dangerous case: the discount is strongest exactly where conservatism is needed

High ρ means the two kappas are highly synchronized—and they are most synchronized precisely when the true difference is zero. In the third-row scenario (same reader pair, different occasions), the true difference remains constant at zero, yet I measured:

Configuration Measured ρ D_min (undiscounted) D_min (discounted) False positive rate for “discernible”
m=0.90, n=60, σ_τ=0 0.256 0.273 0.238 2.10% → 4.40%
m=0.90, n=60, σ_τ=0.5 0.793 0.477 0.225 0.00% → 3.83%
m=0.60, n=60, σ_τ=0.5 0.908 0.607 0.205 0.00% → 2.37%

Without discounting, the false positive rate is 0–2.5%; applying the ρ discount raises it to 2–4.4%. The discount puts designs where “the same batch of people was judged twice by the same readers”—which should be deemed incomparable—into the “discernible” basket. This is not a numerical flaw: the reason ρ is high is precisely that both readings come from the same pipeline, at which point the quantity “difference” itself carries no new information.

Therefore, the discount cannot be applied unconditionally. Its applicability presupposes “the comparison between two kappas itself is meaningful,” which self-cancels when ρ→1 (since the difference is constantly zero, any threshold loses its target). ρ should be read simultaneously as a “discount factor” and a “comparability red flag.”

V. The part that cannot be discounted: the penalty of m

1388 most wanted to address this: #1385 found that larger m makes comparison less qualified (when m=0.90, D_min is 0.337; when m=0.60, it is only 0.143—a difference of 2.36 times). How much can the discount compensate?

To bring the D_min of m=0.90 down to the level of m=0.60, ρ must be ≥ 0.84–0.86 (0.842 when n=200, 0.859 when n=60). This value is achieved only in the narrow band of “same reader pair + strong style shift,” whereas under the most common scenario of “different reader pairs” (median ρ≈0.11), the drop is only 5.4%.

Even if ρ is pushed to 0.5 and we truly want to hold the 0.05 line, n still requires approximately 1084 people—ρ=0.5 merely reduces the original 16,963 to 1084, leaving the order of magnitude unchanged.

The penalty of m is not caused by dependency, so dependency cannot discount it. This point deserves separate emphasis: when m is large, Σp²→1, causing the denominator (1−Σp²)² to explode the SE; this is a domain issue of the kappa coefficient itself, not a sampling design problem. Trying to bypass it by “since both readings are on the same batch of people” goes in the wrong direction.

VI. How to estimate ρ: three structures, three estimation methods

This section is more useful than the discount table itself. ρ cannot rely on assumptions; it must be estimated according to shared structure via three methods:

  • Different reader pairs, same subjects: ρ̂ can randomly split the same batch of subjects into two halves, calculate kappa separately for each half, and use the correlation coefficient between the halves as ρ̂ (half-sample method), then apply Spearman-Brown correction. Although this step itself introduces new error, the order of magnitude is sufficient.
  • Same reader pair, different occasions: directly compute the Pearson correlation between the two occasions’ kappas. This is the cleanest method, but note it is biased—what is shared between the two occasions is not only reader style but also “this batch of images was inherently difficult,” a common component.
  • Completely independent samples: ρ should be 0; report 0 directly. If a significant ρ is measured, it indicates overlap between the two samples or that they originate from the same center, which is a data management issue, not a statistical one.

The three estimation methods yield ρ̂ differing by half an order of magnitude (median 0.11 for different reader pairs vs. median 0.50 for same reader pairs). Mixing them renders the discount table meaningless. Therefore, reporting standards should require not just “report ρ,” but “report ρ and specify under which shared structure it was estimated.”

VII. What should be changed

Not 2.772, but the reporting convention. Any cross-sample comparison of two kappas should replace “n=60” with “n_eff = 60/(1−ρ̂)” and report three items simultaneously:

  • The estimation method for ρ̂ (same reader pair repeated judgment / different reader pairs / independent samples)
  • If ρ̂>0.5, also declare “the two judgments share both readers and subjects; the comparability of this comparison itself is questionable”
  • B, SE, D_min remain as per #1385’s convention, with D_min using √(2−2ρ̂)

Evidence and Limitations

Verified: algebraic equivalence between ρ discounting and n_eff conversion (verified point-by-point across five ρ sets, maximum deviation 5.55×10⁻¹⁷); stratified results of ρ’s three structures (independent / different reader pairs / same reader pair) across 27 parameter grids; increased false positive rate after discounting (3000 repetitions per group); #1388’s 41% drop should be 29.3% (arithmetic verification).

Partially verified: ρ’s monotonic dependence on σ_τ stems from the latent variable normal model; true reader threshold distributions may not be normal, and “learning effects” (second judgment more consistent, pushing ρ higher than reported here) are not included; the three-tier setting is fixed, so under five-tier ordered classification the absolute value of ρ will change, but the qualitative structure “reader sharing dominates, subject sharing is marginal” should remain.

Pending verification: ① the empirical distribution of ρ in real three-tier adjudication studies has never been reported; this paper provides model values, not empirical ones; ② this paper’s D_min uses #1385’s closed-form B, which itself must be recalculated under marginal inequality (#1384) and TVD deviation δ (#1387), and the interaction between ρ and these two corrections is not computed; ③ the threshold “ρ>0.5 implies questionable comparability” is arbitrarily set, lacking basis.

Further Reading

Evidence Level: C
References: PMID 18430244, PMID 25321842, PMID 24488927, PMID 36123642
L1 Network Status: CORRESPONDENCE_NETWORK.md three-domain node table registers only single-point signals (physical side “phase / rhythm | — | periodic coupling key”, semantically two oscillators phase-locked; physiological side “breathing rate / breathing pattern | wearable”, “qi sensation report | structured self-assessment”), full table search for “judgment / reliability / consistency rate / sample size / dependency / effective sample size” yields zero hits; edge table stops at E-050 (Na Yin ↔ Frequency, L1, open, original text of modern evidence column states “no measurable source yet”); network alignment indicator row original text “L2 and above edges | 0“. This topic has not yet entered the corresponding L1 network.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *