Evidence level: C
我们一直以为:两位判读者对「无变化」的占比估得不一样,kappa 的可解释范围就会被撑宽。算下来正好相反——偏倚几乎不动带宽,它砍的是天花板。而那条「带宽超过 0.05 就不许比」的禁令,在真实的三档裁决里永远不会被触发。
正文
上一篇算清了「边际相等」这个前提下的事:给定主档占比 m 与观察一致率 p₀,kappa 的残余自由度收在一条闭区间里,带宽随 m 单调收窄。但那个前提是假设两位判读者对三个档位的分配完全一致。真实场景几乎从不如此。
Byrt 在 1993 年就把这件事拆得很清楚:kappa 不是「一个」一致度,它混着偏倚(bias)、流行度(prevalence)与观察一致率三个可分离的指数(PMID 8501467)。两人对主档占比的估法不同,就是那个偏倚项。所以问题很具体:这个偏倚项 δ 一进来,上篇那条闭区间会变成什么样?
一、先纠正一个直觉:偏倚不是往宽处推
按最自然的参数化:A 判读者把主档判成 m+δ/2,B 判读者判成 m−δ/2,剩余质量两人各自平分给另外两档。此时两位的边际分布之间的总变差距离恰好等于 δ。
于是两件事立刻定死:
一致率的上限被边际锁住。两位判读者的「对角线最多能对上多少」,等于 1 减去两边际的总变差距离,也就是 1−δ。每多 δ 的偏倚,就有 δ 的观测一致率从物理上再也拿不到——这部分一致率被「两人对档位分布的判断不同」这个事实本身扣掉了,跟判得准不准无关。
期望一致率的均值只会往下掉一点点。因为两边的次档质量都被压到 (1−m)/2 附近,Σp₁ₖp₂ₖ 这个点积里出现的是一个 δ² 项:δ=0.05 时,期望一致率只降 0.0009 左右。
两个量一对比,答案就出来了:一致率上限的损失是 δ 的一阶项,期望一致率的损失是 δ 的平方项。δ=0.05 时,上限损失 0.05 是期望一致率损失 0.0009 的 53 倍。所以偏倚进来之后,kappa 的可达区间不是被撑宽,而是整个往下压。
二、数字摆出来
固定 m=0.90(主档占九成),扫 δ:
| δ | kappa 可达区间 | 上限跌幅 |
|---|---|---|
| 0 | [−0.111, 1.000] | — |
| 0.01 | [−0.111, 0.946] | 0.054 |
| 0.02 | [−0.110, 0.892] | 0.108 |
| 0.03 | [−0.108, 0.838] | 0.162 |
| 0.05 | [−0.103, 0.731] | 0.269 |
| 0.08 | [−0.092, 0.573] | 0.427 |
| 0.10 | [−0.081, 0.470] | 0.530 |
下界几乎不动(从 −0.111 到 −0.103,只抬了 0.008),上界掉了 0.269。所以区间长度确实缩短了——从 1.111 缩到 0.835——但缩短的方向是「天花板塌下来」,不是「地板抬上去」。
而 Σp² 的可行域宽度呢?m=0.90 时从 0.00500 变成 0.00469,只缩了 6.2%。m=0.95 时从 0.00125 掉到 0.00094,缩 25%,但绝对量只有 0.0003。
所以「边际一差,带宽就被撑宽」这个担心,量级上根本不成立。 真实的效应是:偏倚把天花板搬低了,而地板原封不动。
三、天花板一塌,0.05 这条线就更荒谬了
上篇定过一条使用限制:带宽超过 0.05 就不许跨档比较。现在可以检查这条线到底什么时候才生效。
Σp² 的可行域宽度有个闭式:在边际相等时等于 (1−m)²/2。它超过 0.05 的条件是 m < 1−√0.1 ≈ 0.684。也就是说,主档占比要低于 68.4% 才会触发。
但三档裁决之所以要报 kappa,恰恰是因为主档占压倒多数(八成、九成)。主档只占 68% 的三档判读,本身就已经不是「大部分人都说没变化」那种情形了,此时 kappa 反而不那么病态。这条线在它最该起作用的区间里,永远不会被触发。 它是个死条款。
更要命的是,偏倚这件事本身对 kappa 的杀伤,远超过 0.05:
| m | kappa 上限跌幅(δ=5pp) | 是 0.05 的几倍 |
|---|---|---|
| 0.75 | 0.123 | 2.5× |
| 0.80 | 0.147 | 2.9× |
| 0.85 | 0.187 | 3.7× |
| 0.90 | 0.269 | 5.4× |
| 0.95 | 0.514 | 10.3× |
主档占九成五时,仅仅 5 个百分点的偏倚就能让 kappa 上限掉一半。这个量级下,讨论「带宽 0.05」已经毫无意义——你连天花板在哪都不确定,谈区间宽度是奢侈。
四、反解:从报出来的 kappa 倒推偏倚
把这关系翻过来用更有价值。给定 m 与观测到的 κ,可以反推「这份数据里至少存在多大的偏倚」:
δ ≥ (1 − Σp²₀)(1 − κ)
m=0.90 时报 κ=0.85,说明至少存在 2.8 个百分点的偏倚;报 κ=0.60,至少 7.4 个百分点。这意味着:一个漂亮的 kappa 读数,本身就是「两人边际相近」的证据——因为你不可能在偏倚很大的情况下拿到高 kappa,天花板不允许。
这条反解有两处实用价值。第一,它是免费的内部一致性检查:如果研究同时报了两位判读者的主档占比,算出 δ,代进去看 κ 是否超过上限,超了就是数据有错。第二,它给出报告门槛:δ 超过 δ_tol = 0.05(1−Σp²₀) 时,κ 的绝对值已经没法和别的研究比了,只能做同一批数据内部的相对比较。这个门槛在 m=0.90 时是 0.9 个百分点,m=0.95 时是 0.5 个百分点——比 0.05 那条带宽线严格 5 到 10 倍。也就是说,一个真正起作用的门槛,远比现在这条宽松的线苛刻。
五、这对「三档裁决」意味着什么
上篇的结论要改一处措辞。原文说「边际不等情形下的 Σp² 可行域尚未推出闭式,只知道它比相等情形宽」——现在有了:它不宽,它更窄,窄的幅度是 δ² 量级,小到可以忽略。 真正被偏倚改变的不是可行域,是一致率上限。
Hoehler 当年处理那两个悖论的思路(PMID 8621993)本质上也是这一条:kappa 的病不在分母,在分子被边际锁死。Byrt 三指数分解(PMID 8501467)与后来多系数比较的研究(PMID 25168681、PMID 23627889)反复确认的也是同一件事——任何单一一致性系数都同时被边际与真实一致度两个东西驱动,不拆开就没法读。
所以三档裁决的报告规范应该改成:报 (m, δ, p₀, κ) 四元组,而不是上篇说的三元组。δ 这一项不是补充说明,它是决定 κ 天花板的那一项。缺了它,κ 的绝对值不可跨研究比较;有了它,至少能判断这份 κ 是在天花板附近还是在半山腰。
证据与局限
已验证:
– 偏倚 δ 与 κ 上限的关系 δκmax = δ/(1−Σp²₀) 是代数恒等式(略去 δ² 二阶项后误差 <2%),本文所有表格均由脚本实算
– Σp² 可行域宽度在偏倚下只缩不宽,缩幅为 δ² 量级(m=0.90、δ=0.05 时仅 6.2%)
– 上界 1−δ 来自运输多面体的 max trace = 1 − TVD,这一步是精确的而非近似
部分验证:
– 「真实三档裁决中 δ 的典型值」未知。本文所有结论按 δ 扫参数给出,但缺一个实测的 δ 分布。若真实 δ 普遍小于 1 个百分点,则天花板效应可忽略、上篇三元组够用;若普遍在 5 个百分点以上,则四元组是硬要求
– Byrt 三指数分解(PMID 8501467)与多系数比较(PMID 25168681)支持「边际必须单列报告」这一方向,但都没有给出本文这个具体的反解式
待验证:
– 本文假设次档质量在两位判读者间按剩余质量等分。若真实情形是「一人把剩余全给第二档、另一人全给第三档」,则 TVD 会大于 δ,上界会被压得更低。这个偏离有多大需要实测
– 加权 kappa 下 δ 的作用是否同样是一阶压低上界,没有算过
– 四方桌以上的档数时,δ 与 TVD 不再相等,本文的 δ→上限映射需要重推
相关阅读
- 报出 kappa、一致率和主档占比,还剩多少没说:那条带宽就是辨识的天花板
- kappa 其实只有一个自变量:观察一致率 0.90 能对应负 0.11 到 0.85 之间的任何值
- 观察一致率九成,kappa 却能掉成负数:三档裁决的崩塌曲线
- 共振意识场研究池
证据等级: C
参考文献: PMID 8501467, PMID 2348207, PMID 8621993, PMID 25168681
L1 网络状态:CORRESPONDENCE_NETWORK.md 三域节点表仅登记单点信号(物理侧「相位 / 节律 | — | 周期性耦合的关键」,语义为两振荡锁相、不带一致性/判读/边际/偏倚语义;生理侧「呼吸率 / 呼吸模式 | 可穿戴」「气感报告 | 结构化自评」),全表检索「判据 / 裁决 / 一致率 / 判读 / 信度 / 边际 / 观察者间 / 偏倚 / 带宽」全部零命中;边表止于 E-050(纳音↔频率,L1、open,现代证据栏原文「尚无任何可测来源」);网络对齐指标行原文「L2 以上边数 | 0」。→ 该主题尚未进入 L1 对应网络,本文封顶 C。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Two adjudicators say the main archive’s proportion difference is five points, and what is stretched wide is not bandwidth but ceiling: the 0.05 line is actually a dead clause.
Evidence level: C
We have long assumed: if two raters disagree on the proportion of the “no-change” category, the range of kappa’s interpretability must widen. The calculation shows the opposite—bias hardly affects the bandwidth; it cuts the ceiling. Consequently, the rule “do not compare across categories if bandwidth exceeds 0.05” will never be triggered in real three-tier adjudications.
Body
The previous post resolved the case under the assumption of “equal margins”: given the main-category proportion m and the observed agreement rate p₀, kappa’s residual degrees of freedom lie within a closed interval, with bandwidth shrinking monotonically as m increases. That premise, however, assumes the two raters allocate the three tiers identically—a condition rarely met in practice.
Byrt (1993) already disentangled this clearly: kappa is not “one” measure of agreement; it combines three separable indices—bias, prevalence, and observed agreement (PMID 8501467). When the two raters differ in their estimate of the main-category proportion, that difference constitutes the bias term. Thus the question becomes specific: how does introducing this bias term δ alter the closed interval derived earlier?
I. First, correct an intuition: bias does not widen the range
Using the most natural parameterization: Rater A assigns the main category proportion as m + δ/2, while Rater B assigns it as m − δ/2; the remaining mass is split equally between the two raters across the other two tiers. Under this setup, the total variation distance between their marginal distributions equals exactly δ.
Two consequences follow immediately:
The upper bound of the agreement rate is locked by the margins. The maximum possible overlap on the diagonal—how much the two raters can agree given their differing marginal distributions—is 1 minus the sum of the total variation distances, i.e., 1 − δ. Each additional δ of bias removes δ of the observed agreement rate from physical possibility—this portion of agreement is lost simply because “the two raters disagree on the distribution of categories,” regardless of how accurately they judge.
The mean of the expected agreement rate drops only slightly. Because the secondary-tier masses are compressed near (1−m)/2, the dot product Σp₁ₖp₂ₖ contains a δ² term: when δ = 0.05, the expected agreement rate falls by only ~0.0009.
Comparing these two effects yields the answer: the loss in the upper bound is a first-order term in δ, whereas the loss in the expected agreement rate is a second-order (δ²) term. At δ = 0.05, the upper-bound loss of 0.05 is 53 times larger than the expected-agreement loss of 0.0009. Therefore, once bias enters, kappa’s reachable interval is not widened; it is pressed down as a whole.
II. Numbers laid out
Fixing m = 0.90 (main category accounts for 90%), we sweep δ:
| δ | kappa reachable interval | upper-bound drop |
|---|---|---|
| 0 | [−0.111, 1.000] | — |
| 0.01 | [−0.111, 0.946] | 0.054 |
| 0.02 | [−0.110, 0.892] | 0.108 |
| 0.03 | [−0.108, 0.838] | 0.162 |
| 0.05 | [−0.103, 0.731] | 0.269 |
| 0.08 | [−0.092, 0.573] | 0.427 |
| 0.10 | [−0.081, 0.470] | 0.530 |
The lower bound remains nearly unchanged (from −0.111 to −0.103, a rise of only 0.008), while the upper bound drops by 0.269. Thus, the interval length indeed shrinks—from 1.111 to 0.835—but the direction of shrinkage is “the ceiling collapses,” not “the floor rises.”
What about the width of the feasible region for Σp²? When m = 0.90, it contracts from 0.00500 to 0.00469—a reduction of only 6.2%. At m = 0.95, it falls from 0.00125 to 0.00094, a 25% drop, yet the absolute amount is merely 0.0003.
Therefore, the worry that “a difference in margins widens the bandwidth” is quantitatively unfounded. The real effect is that bias lowers the ceiling, while the floor remains untouched.
III. Once the ceiling collapses, the 0.05 line becomes absurd
The previous post established a usage restriction: do not compare across categories if the bandwidth exceeds 0.05. Now we can check when this line actually activates.
The feasible width of Σp² has a closed-form expression: under equal margins it equals (1−m)²/2. This width exceeds 0.05 only when m < 1 − √0.1 ≈ 0.684. That is, the main-category proportion must be below 68.4% for the threshold to trigger.
Yet three-tier adjudications report kappa precisely because the main category dominates (80%, 90%). A three-tier judgment where the main category accounts for only 68% is already not a case of “most people say no change”; in such a scenario, kappa becomes less pathological. This threshold never activates in the very region where it should matter. It is a dead clause.
Even more critically, the damage bias inflicts on kappa far exceeds 0.05:
| m | kappa upper-bound drop (δ = 5pp) | multiple of 0.05 |
|---|---|---|
| 0.75 | 0.123 | 2.5× |
| 0.80 | 0.147 | 2.9× |
| 0.85 | 0.187 | 3.7× |
| 0.90 | 0.269 | 5.4× |
| 0.95 | 0.514 | 10.3× |
When the main category accounts for 95%, a mere 5-percentage-point bias halves kappa’s upper bound. At this scale, discussing “bandwidth 0.05” is meaningless—you cannot even determine where the ceiling lies, so talking about interval width is extravagant.
IV. Inverting: deriving bias from reported kappa
Reversing this relationship proves more valuable. Given m and the observed κ, one can infer the minimum bias present in the data:
δ ≥ (1 − Σp²₀)(1 − κ)
If m = 0.90 and the reported κ = 0.85, this implies at least 2.8 percentage points of bias; if κ = 0.60, at least 7.4 percentage points. This means: a high kappa reading itself serves as evidence that the two raters’ margins are close—because you cannot achieve a high kappa when bias is large, as the ceiling forbids it.
This inversion has two practical uses. First, it provides a free internal consistency check: if a study reports both raters’ main-category proportions and computes δ, substituting it into κ should not exceed the upper bound; exceeding it indicates data error. Second, it sets a reporting threshold: when δ exceeds δ_tol = 0.05(1 − Σp²₀), the absolute value of κ can no longer be compared across studies; it can only serve for relative comparison within the same dataset. This threshold is 0.9 percentage points when m = 0.90 and 0.5 percentage points when m = 0.95—five to ten times stricter than the 0.05 bandwidth line. Thus, any meaningful threshold is far more stringent than the currently proposed loose one.
V. What this means for “three-tier adjudication”
The conclusion of the previous post requires one wording adjustment. The original stated: “Under unequal margins, the feasible region for Σp² has no closed-form expression; we only know it is wider than under equal margins.” Now we have: it is not wider; it is narrower, by an amount of δ² order, so small as to be negligible. What bias actually alters is not the feasible region but the upper bound of agreement.
Hoehler’s approach to those two paradoxes (PMID 8621993) essentially follows this line: kappa’s pathology lies not in the denominator but in the numerator being locked by the margins. Byrt’s three-index decomposition (PMID 8501467) and subsequent multi-coefficient comparison studies (PMID 25168681, PMID 23627889) repeatedly confirm the same point—any single agreement coefficient is driven simultaneously by margins and true agreement; without separating them, interpretation is impossible.
Therefore, the reporting standard for three-tier adjudication should be revised to: report the quadruple (m, δ, p₀, κ) rather than the triple proposed earlier. The δ term is not supplementary; it determines kappa’s ceiling. Without it, the absolute value of κ cannot be compared across studies; with it, at least one can judge whether this κ lies near the ceiling or midway.
Evidence and Limitations
Verified:
– The relationship between bias δ and kappa’s upper bound, δκmax = δ/(1−Σp²₀), is an algebraic identity (neglecting the δ² second-order term introduces <2% error); all tables in this paper were computed via script.
– The feasible width of Σp² under bias shrinks rather than widens, by an amount of δ² order (only 6.2% when m = 0.90 and δ = 0.05).
– The upper bound 1 − δ stems from the max trace of the transport polytope equaling 1 − TVD; this step is exact, not approximate.
Partially verified:
– The “typical value of δ in real three-tier adjudications” remains unknown. All conclusions here are presented by sweeping δ as a parameter, but lack an empirical δ distribution. If real δ is generally under 1 percentage point, the ceiling effect can be ignored and the previous triple suffices; if δ is commonly above 5 percentage points, the quadruple becomes mandatory.
– Byrt’s three-index decomposition (PMID 8501467) and the multi-coefficient comparison studies (PMID 25168681) support the direction that “margins must be reported separately,” but none provide the specific inversion formula presented here.
To be verified:
– This paper assumes that secondary-tier mass is split equally between the two raters. If reality is “one rater assigns all remaining mass to tier 2, the other to tier 3,” then TVD would exceed δ and the upper bound would be pressed even lower. How large this deviation is requires empirical measurement.
– Under weighted kappa, does δ still exert a first-order suppression of the upper bound? This has not been calculated.
– For four or more tiers, δ and TVD are no longer equal; the mapping δ → upper bound presented here must be rederived.
Related Reading
- Reporting kappa, agreement rate, and main-category proportion leaves much unsaid: that bandwidth is the ceiling of interpretability
- Kappa actually has only one independent variable: observed agreement rate 0.90 can correspond to any value between −0.11 and 0.85
- Observed agreement at 90%, yet kappa plummets to negative: the collapse curve of three-tier adjudication
- Resonance consciousness field research pool
Evidence level: C
References: PMID 8501467, PMID 2348207, PMID 8621993, PMID 25168681
L1 network status: The CORRESPONDENCE_NETWORK.md three-domain node table registers only single-point signals (physical side “phase / rhythm | — | periodic coupling key”, semantically meaning two oscillators phase-locked, without agreement/adjudication/margin/bias semantics; physiological side “breathing rate / breathing pattern | wearable”, “qi-sensation report | structured self-assessment”). A full-table search for “criterion / adjudication / agreement rate / rating / reliability / inter-observer / bias / bandwidth” yields zero hits; the side table stops at E-050 (Na Yin ↔ frequency, L1, open, original modern evidence column states “no measurable source yet”); the network alignment indicator row reads “L2+ edges | 0”. → This topic has not yet entered the L1 corresponding network; this paper caps at C.
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply