一致性天花板不必事后求解:把加权 kappa 的可达区间写成闭式,判读设计在招募前就能公示上限

Written by

in

Language

Evidence level: C

同一对边际下,加权 kappa 可以是零也可以接近一:这个留白能不能不靠逐例解规划,而是由边际和档数三个数直接算出来,让「天花板」在收材料之前就公示?

正文(约1250字)

前一篇(#1433)用线性规划求得了硬结论:固定两位判读者各自的边际分布(行和 r、列和 c),同一批数据下加权 kappa 的取值是一个区间而不是一个点——三档线性权下区间是 [-0.4496, +0.9017],二次权 [-0.8182, +0.9301];档数越多区间越宽,四档宽 1.3924、五档宽 1.9381。端点与三十万次抽样极值吻合到小数点后三位。但这批端点全部是逐例解出来的:每换一对边际,就要重新解一遍规划。判读设计者拿到自己的边际预估后,没有一张「查表即得」的图。

这就引出一个纯解析问题:把「行和等于 r、列和等于 c、每格非负」约束下的加权 kappa 最大值(以及最小值)写成闭式。

第一步是换个视角。固定边际求加权重合最大,本质是一个运输问题:把行边际的「质量」分给列边际的「仓库」,每格装多少由列和约束决定,目标函数是权重矩阵 w_ij 与分配量的点乘。运输问题有成熟的对偶理论——最大重合等于一组影子价格下「行势 + 列势不超过 w_ij」约束下的最紧和。而这里的权重矩阵 w_ij = 1 − (档位差绝对值)/(k−1) 不是普通矩阵:它沿两个方向都单调,行内相邻格的权重差是常数,这就是优化文献里经典的 Monge 结构。Monge 结构下运输问题的最优解不需要通用规划器,贪心配对即可,端点因此有望写成边际的显式函数。

第二步是真正要裁决的问题:端点到底依赖边际的什么。如果端点能被「边际的一阶矩 + 档数 k」三个数完全确定(相对误差 < 1%),那么判读设计在招募前就可以公示上限:凡报告的有序一致性超过公示上限,直接判数据错误,不必再查。这一步有直接的文献呼应——κ 对边际失衡的敏感性是三十年老问题:Feinstein 与 Cicchetti 列出的两大悖论(2348207)核心就是同一列联表在边际不同时 κ 可以差一个数量级;Byrt 等给出按边际可预计算的最大 κ(8501467),证明「边际定上限」这件事在二类情形早已是闭式;Warrens 在有序多类情形系统比较了加权系数的分辨力差异(25168681),并强调多读者有序分类必须显式声明权重族(26095449)——权重族定了,Monge 结构才成立,闭式才有唯一答案。

若裁决走向另一分支——端点依赖边际的高阶矩——那么「上限」一词就不能随意使用:设计文档至少须报告边际的前三阶矩,否则公示的就是一个不存在的数。

三分支判据如下:其一,三点确定(一阶矩 + 档数,误差 < 1%)⇒ 上限查表化,招募前公示,超限即判数据错误,审查流程从「事后重算」降为「对表」;其二,须前三阶矩 ⇒ 上限仍可公示,但必须附边际矩报告义务,缺报告的研究其一致性数值视为「无可核验区间」;其三,端点无低维充分统计量表示 ⇒ 放弃闭式公示路线,退回逐例规划,但此时应把 #1433 的规划端点程序打包成标准工具,让「同一对边际下 kappa 区间宽度」成为任何一致性报告的必填字段。

回到桥接场景:本组将来做脉象档位、气味等级这类有序判读的重复测量时,判读者边际几乎必然偏斜(主档占大头)。偏斜边际正是区间最宽的地带——同样的 kappa 0.6,在偏斜边际下可能对应「几乎不可能更好」也可能对应「还能差出 0.9」。天花板闭式就是把这个歧义在招募前钉死:设计阶段先算上限,实验结束看数值离天花板多远,而不是拿一个裸 kappa 和 0.8 的约定俗成线比较。

证据与局限

已验证:固定边际下加权 kappa 是区间而非点(#1433 线性规划 + 三十万次抽样);κ 对边际失衡敏感、有悖论(Feinstein & Cicchetti 2348207);二类情形 max κ 可由边际预计算(Byrt 8501467);权重族显式声明是多读者有序分类的必要条件(26095449)。

部分验证:Monge 结构使运输问题有贪心最优解是优化理论成熟结果,但「本权重矩阵端点能否写成边际低阶矩的显式函数」尚未有人对本组 #1433 的具体端点数值做过拟合检验。

待验证:三点闭式(一阶矩+档数,误差<1%)在三档/四档/五档下是否同时成立;高阶矩分支下「前三阶矩」是否是充分而非仅必要条件;公示上限后「超限即判数据错误」的操作阈值如何与抽样波动(置信区间半宽)相容。

相关阅读

证据等级: C
参考文献: PMID 2348207, PMID 8501467, PMID 26095449, PMID 25168681

证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索

L1 网络状态:该主题尚未进入 L1 对应网络。EDGE_TABLE / CORRESPONDENCE_NETWORK 中无「加权 kappa 可达区间闭式 × 运输对偶 × Monge 结构」对应边;最近邻为「E-060 标签前提」边(KB-292/KB-317 两处 conflict:标签信度低 + 样本量单位错配 + 加权系数精度区间错配),为相邻方法学背景、非同一条边,本篇正文不据此写成已验证机制,全文封顶 C 级。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Consistency Ceiling Without Post-hoc Solution: Closed-form Reachability Interval of Weighted Kappa for Pre-recruitment Design Limit Disclosure

Evidence level: C

Under the same pair of marginal distributions, weighted kappa can be zero or approach one: could this gap be determined not by solving the plan case-by-case, but directly from the marginals and the number of categories, thereby publishing the “ceiling” before data collection begins?

Body (approx. 1250 words)

The previous entry (#1433) used linear programming to derive a hard conclusion: fixing the marginal distributions of two raters (row sums $r$ and column sums $c$), the value of weighted kappa under the same dataset is an interval rather than a single point—under three-tier linear weighting the interval is $[-0.4496, +0.9017]$, under quadratic weighting $[-0.8182, +0.9301]$; the more categories, the wider the interval, with four tiers spanning 1.3924 and five tiers spanning 1.9381. These endpoints align with the extremes of thirty million random samples to three decimal places. However, all these endpoints were derived by solving the plan case-by-case: every change in the pair of marginals requires re-solving the optimization. Designers of the rating protocol, once they have their own marginal estimates, possess no “look-up table” chart.

This raises a purely analytical problem: express the maximum (and minimum) of weighted kappa under the constraints “row sums equal $r$, column sums equal $c$, and every cell non-negative” in closed form.

The first step is to shift perspective. Maximizing weighted agreement under fixed marginals is essentially a transportation problem: allocating the “quality” of row marginals to the “warehouses” of column marginals, with each cell’s allocation determined by the column sum constraint, and the objective function being the dot product of the weight matrix $w_{ij}$ and the allocation. Transportation problems have mature dual theory—the maximum overlap equals the tightest sum under constraints of “row potential + column potential not exceeding $w_{ij}$” under a set of shadow prices. Here, the weight matrix $w_{ij} = 1 – \frac{|k_i – k_j|}{k-1}$ is not an ordinary matrix: it is monotonic in both directions, with constant weight differences between adjacent cells within a row. This is the classic Monge structure found in optimization literature. Under Monge structure, the optimal solution to the transportation problem does not require a general planner; greedy pairing suffices, making it possible to express the endpoints as explicit functions of the marginals.

The second step addresses the core question: exactly what properties of the marginals determine these endpoints? If the endpoints can be fully determined by “the first moment of the marginals + the number of categories $k$” (with relative error < 1%), then the design of the rating protocol can publish the upper limit before recruitment: any reported ordered consistency exceeding the published ceiling is immediately judged as data error, without further checking. This step has direct literature support—the sensitivity of $\kappa$ to marginal imbalance is a thirty-year-old problem: the two paradoxes listed by Feinstein and Cicchetti (2348207) center on the fact that the same contingency table can yield $\kappa$ differing by an order of magnitude when marginals differ; Byrt et al. provided the maximum $\kappa$ precomputable from marginals in the two-tier case (8501467), proving that “margins determine the ceiling” is already a closed-form result in the binary case; Warrens systematically compared the resolving power of weighted coefficients in ordered multi-tier cases (25168681), emphasizing that the weight family must be explicitly declared for multi-rater ordered classification (26095449)—once the weight family is fixed, the Monge structure holds, and the closed-form solution is unique.

If the verdict follows another branch—the endpoints depend on higher-order moments of the marginals—then the term “ceiling” cannot be used arbitrarily: the design document must at least report the first three moments of the marginals, otherwise the published ceiling is a non-existent number.

The three-branch criteria are as follows: first, if three points suffice (first moment + number of categories, error < 1%) ⇒ the ceiling becomes a look-up table, published before recruitment, and any excess immediately judges data error, reducing the review process from “post-hoc recalculation” to “table lookup”; second, if three moments are required ⇒ the ceiling can still be published, but the design document must include an obligation to report marginal moments, and studies lacking such reports have their consistency values treated as “unverifiable intervals”; third, if the endpoints lack a low-dimensional sufficient statistic representation ⇒ abandon the closed-form publication route and revert to case-by-case planning, but in this case, the planning endpoint program from #1433 should be packaged as a standard tool, making “the width of the kappa interval under the same pair of marginals” a mandatory field in any consistency report.

Returning to the bridging scenario: when this group conducts repeated measurements of ordered ratings such as pulse wave categories or scent intensity levels, the marginal distribution of raters will almost certainly be skewed (the main category dominates). Skewed marginals correspond precisely to the widest region of the interval—the same $\kappa$ of 0.6 might correspond either to “almost impossible to improve” or to “still could be worse by 0.9.” The closed-form ceiling fixes this ambiguity before recruitment: during the design phase, calculate the upper limit; after the experiment, see how far the numerical value lies from the ceiling, rather than comparing a bare $\kappa$ against the conventional 0.8 threshold.

Evidence and Limitations

Verified: Under fixed marginals, weighted kappa is an interval not a point (#1433 linear programming + thirty million random samples); $\kappa$ is sensitive to marginal imbalance, exhibiting paradoxes (Feinstein & Cicchetti 2348207); the maximum $\kappa$ in the two-tier case can be precomputed from marginals (Byrt 8501467); explicit declaration of the weight family is necessary for multi-rater ordered classification (26095449).

Partially Verified: Monge structure ensures greedy optimality for transportation problems, a mature result in optimization theory, but whether “the endpoints of this weight matrix can be expressed as explicit functions of low-order moments of the marginals” has not yet been fitted against specific endpoint values from this group’s #1433.

Pending Verification: Whether the three-point closed form (first moment + number of categories, error < 1%) holds simultaneously under three-tier, four-tier, and five-tier cases; whether “the first three moments” constitute sufficient conditions rather than merely necessary ones under the higher-order moment branch; how the operational threshold of “exceeding the ceiling implies data error” reconciles with sampling fluctuation (confidence interval half-width).

Related Reading

Evidence Level: C
References: PMID 2348207, PMID 8501467, PMID 26095449, PMID 25168681

Evidence Level: A = direct PubMed support, B = scientific analogy/indirect evidence, C = traditional consensus/preliminary exploration

L1 Network Status: This topic has not yet entered the L1 corresponding network. There is no edge in EDGE_TABLE / CORRESPONDENCE_NETWORK linking “closed-form of weighted kappa reachable interval × transportation dual × Monge structure”; the nearest neighbor is the “E-060 label premise” edge (KB-292/KB-317 two conflicts: low label credibility + mismatched sample unit scale + mismatched weighted coefficient precision interval). As this is an adjacent methodological background and not the same edge, the main body of this paper does not write it as a verified mechanism, capping the entire text at Level C.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *