那个决定门槛高低的系数不是猜的:它可由读者各自的边际直接算出,而它并不只是 p_e 的函数

Written by

in

Language

Evidence level: C

门槛公式 n ≈ C × (39.2/(1−p_e))² 里唯一被当成「数值读数」的 C,其实有闭式解;但把它化简成 p_e 的单变量函数会引入两到四倍半的误差,因为真 C 由两个读者各自的整条边际共同决定。

正文

一、问题的形状:一个被当成读数的数

先接上一篇的尾巴。#1409 把判读一致性天花板的样本量门槛压成了一条合式:所需例数约等于 C 乘以(39.2 除以 1 减 p_e)的平方。其中放大因子 1/(1−p_e) 是精确代数,没有一个符号是估的;但 C 只是五个 m̄ 取值上的数值读数——0.60 与 0.75 附近约 0.42,0.90 约 0.25,0.95 约 0.18。这篇要回答的是:C 到底是什么的函数。

二、C 是两读者边际的直接函数

第一步先写下 C 的定义。TVD 是两读者各档比例的绝对差之和的一半,在大样本下它的抽样波动可以线性化:每一档的系数取该档差值的符号的一半,于是 C 等于这个符号向量在「两读者比例协方差矩阵之和」上的二次型。符号向量与协方差矩阵都只依赖两个读者的比例本身,所以 C 从来不是自由参数——它是两个边际的直接函数。真正的问题因此不是「C 能不能算」,而是「算它需要几个数」。

三、三个单参数简化式全部失败

这里有个陷阱。逻辑上最自然的做法是假设 C 只由 p_e 决定,毕竟门槛公式里 p_e 是唯一出现在分母上的分布量;文献也反复强调 κ 的标准误依赖边际,但往往把「边际」当成一个笼统的难度旋钮。我们把三类单参数简化式都试了:只用读者一的主档、用两读者主档的平均、只用 1 减 p_e。在 K 从 2 到 6、p_e 从 0.36 到 0.90、真实 TVD 从 0 到 0.20、差落在近端或远端或两档分摊的三格局共 360 个格点上,这些简化式落在真值正负一成以内的只有 12% 到 19%;中位绝对偏差三成到五成,最坏处偏差四点五倍。换句话说,用 p_e 单变量去代 C,误差不是小数点后的事。

四、零差档的折叠正态修正

第二个陷阱更隐蔽。朴素线性化有个失效点:当两读者在某一档上的真实比例完全相等时,该档的符号为零,于是这一档的抽样波动被整个抹掉。但「真实相等」并不意味着「观测相等」——这一档的观测差在零附近来回跳,绝对值并不为零,它贡献的是折叠正态的方差:零点二五乘以(1 减 2/π)再乘该档方差。我们按这个修正重写了 C,得到一条只依赖两读者全部边际的闭式。在 54 个格点、每格五万一千二百例、一万次重抽样的核对中,闭式与蒙特卡洛的相对误差中位不到百分之一,最大 2.5%,全部落在 5% 以内。

五、C 的真身:边际形状与差落点的联合函数

修正之后,C 的结构就清楚了。它既不是 p_e 的函数,也不是 m̄ 的函数,而是「边际形状 + 差落点」的联合函数。固定 p_e 与真实 TVD 完全不变、只改差落在哪几档,C 的跨度在 K=3 时可达 1.32 倍,K=6 时 1.19 倍。p_e=0.65、K=5、TVD=0.10 的例子更直观:差全落在一条主档与一条末档之间时 C 是 0.2346,门槛约两千九百例;同样的边际、同样的 TVD,但差由两条档各分摊一半时 C 升到 0.2823,门槛三千五百例。两个真实分歧程度完全一样的研究,仅因为「分歧集中在一条线上」还是「摊在两条线上」,所需样本量差两成。

那为什么 #1409 那五个 m̄ 读数看起来单调下降、很规整?因为那五个点全是同一种构型——差集中在主档与次档之间,而边际也随之从平缓变尖锐。单调下降是「构型固定、只扫边际」的产物,不是 C 的内在律。一旦构型换掉,同样的 p_e 下 C 可以反向。这解释了低一致区那处反常:0.60 与 0.75 平在同一水平,不是测量噪声,而是两个机制在此交点处互相抵消——边际越平缓,逐档比例越接近,符号向量的非零分量越多,方差上界抬高;而同时各档差值变小,真实波动收窄。

六、这对实践是减法

这条结果对实践的含义是减法而非加法。第一,门槛可以在招募前算出来,但前提是报全两个读者各自的期望边际分布,而不是只报一个 p_e 或一个 κ 目标值——这一点与方法学文献的要求一致:按目标精度反解样本量时必须指定完整边际,只给系数无法定 n。第二,任何「用小样本试点估一个 C 再外推」的路线都不成立,因为 C 随构型的敏感度与 n 无关,试点给不出构型信息。第三,若实践上只能给一个数,那这个数应当按最不利构型取上界——在本轮网格里,两档分摊构型普遍给出更高的 C,用它做保守估计不会低估门槛。

证据与局限

已验证(纯代数与蒙特卡洛,可复现):TVD 的线性化方差只依赖两读者边际,符号向量与协方差矩阵可显式写出;零差档必须按折叠正态补 0.25×(1−2/π)×方差项,否则朴素线性化系统性低估,零差档数从零增到三时低估幅度中位从 0% 增至 12.4%,最坏 17%;修正后的闭式在 54 格点、五万例规模下与仿真相对误差中位 0.9%、最大 2.5%。

部分验证:C 对「差落点」的敏感度(同 p_e、同 TVD 下跨 1.14 至 1.32 倍)只在三档构型(远端单一、近端单一、两档分摊)上扫过,未穷举全部支撑集;「两档分摊构型给出更高 C」这一规律在 K=3 到 6、p_e=0.40 到 0.80 的九个格点上一致,但未证明对所有构型成立,只能作保守上界的经验依据而非定理。

待验证:真实判读研究里「差落点」的分布从未被系统报告过——已发表研究普遍只给 κ、AC1 与 95% 区间,很少给两读者各自的完整边际,因此无法反推构型;这既是本条结论的适用边界,也是下一项工作的入口。此外 C 的闭式在 K 大于 6 或存在结构性零格(某档两读者都从未使用)时是否仍成立,本轮未测。

L1 网络状态:L1 边表与对应网络文件中无「判读一致性度量的方差结构」这条边。最接近的是 E-060 标签前提(状态 conflict:PMID 30383398 报中医医师间诊断一致性在开放人群中为 low、作者称「令人担忧」;KB-317/334 报 AC2 已在真实临床场景跑通、并指出「患病率 ≠ 评定者所见患病率」的实操坑;KB-333 报 κ 型系数对序数/名义的敏感性方向与 Gwet 一系相反),该边只登记「标签信度够不够用」,未登记「信度指标的方差由什么决定」。故本文封顶 C,且不主张任何已验证机制。该主题尚未进入 L1 对应网络,缺口已登记。

相关阅读

证据等级: C
参考文献: PMID 11512137, PMID 22560852, PMID 36123642, PMID 38283651, PMID 23627889, PMID 25168681, PMID 15733050


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

The coefficient that determines the threshold is not guessed: it can be calculated directly from each reader’s marginal utility, and it is not merely a function of p_e.

Evidence level: C

In the threshold formula $n \approx C \times (39.2/(1-p_e))^2$, the only term treated as a “numerical reading” is $C$. However, $C$ actually possesses a closed-form solution; simplifying it into a single-variable function of $p_e$ introduces an error ranging from two to four and a half times, because the true $C$ is jointly determined by the entire margin of each of the two readers.

Main Text

I. The Shape of the Problem: A Number Treated as a Reading

To continue from the previous section. Reference #1409 compressed the sample size threshold for judging consistency into a tractable form: the required number of cases is approximately equal to $C$ multiplied by the square of (39.2 divided by 1 minus $p_e$). Among these, the amplification factor $1/(1-p_e)$ is exact algebra; no symbol here is estimated. However, $C$ itself is merely a numerical reading taken from five values of $\bar{m}$—approximately 0.42 near 0.60 and 0.75, approximately 0.25 near 0.90, and approximately 0.18 near 0.95. This paper addresses the question: what is $C$ actually a function of?

II. $C$ is the Direct Function of Two Readers’ Margins

The first step is to write down the definition of $C$. TVD (Traditional Consensus) is half the sum of absolute differences between the proportions of each category for the two readers. Under large sample conditions, its sampling fluctuation can be linearized: the coefficient for each category takes half the sign of the difference in that category. Thus, $C$ equals the quadratic form of this sign vector on “the sum of the two readers’ proportion covariance matrices.” Both the sign vector and the covariance matrix depend solely on the proportions of the two readers themselves. Therefore, $C$ is never a free parameter—it is a direct function of the two margins. The real problem is therefore not “Can $C$ be calculated?” but “How many numbers are needed to calculate it?”

III. All Three Single-Parameter Simplifications Fail

Here lies a trap. Logically, the most natural assumption is that $C$ depends only on $p_e$, since in the threshold formula $p_e$ is the only distributional quantity appearing in the denominator; literature also repeatedly emphasizes that the standard error of $\kappa$ depends on margins, yet often treats “margins” as a generic difficulty knob. We tested all three types of single-parameter simplifications: using only the primary category of Reader 1, using the average of the primary categories of both readers, and using only $1 – p_e$. Across 360 grid points where $K$ ranges from 2 to 6, $p_e$ ranges from 0.36 to 0.90, true TVD ranges from 0 to 0.20, and differences fall in near-end, far-end, or split-between-two-categories configurations, these simplifications fall within $\pm10\%$ of the true value only 12% to 19% of the time; the median absolute deviation ranges from 30% to 50%, and in the worst case, the error reaches 4.5 times. In other words, substituting $C$ with a single-variable function of $p_e$ introduces errors far beyond mere decimal precision.

IV. Folded Normal Correction for Zero-Difference Categories

The second trap is more subtle. Naive linearization has a failure point: when the true proportions of two readers are exactly equal in a certain category, the sign for that category becomes zero, and thus the sampling fluctuation of that category is entirely erased. However, “true equality” does not imply “observed equality”—the observed difference in that category fluctuates around zero, and its absolute value is not zero; it contributes the variance of a folded normal distribution: $0.25 \times (1 – 2/\pi) \times$ the variance of that category. Rewriting $C$ according to this correction yields a closed-form expression depending solely on the complete margins of both readers. In verification against 54 grid points, each with 51,200 cases and 10,000 Monte Carlo resamples, the median relative error between the closed form and Monte Carlo is less than 1%, with a maximum of 2.5%; all errors fall within 5%.

V. The True Nature of $C$: A Joint Function of Margin Shape and Difference Location

After correction, the structure of $C$ becomes clear. It is neither a function of $p_e$ nor of $\bar{m}$; rather, it is a joint function of “margin shape + difference location.” Holding $p_e$ and true TVD constant while varying only the location of the differences, the span of $C$ reaches 1.32 times when $K=3$ and 1.19 times when $K=6$. The example where $p_e=0.65$, $K=5$, and TVD=0.10 is particularly illustrative: when differences fall entirely between one primary category and one terminal category, $C$ is 0.2346, yielding a threshold of approximately 2,900 cases; with the same margins and TVD but differences split equally between two categories, $C$ rises to 0.2823, yielding a threshold of 3,500 cases. Two studies with identical true disagreement levels differ by 20% in required sample size solely because the “disagreement concentrates on one line” versus “spreads across two lines.”

Why do the five $\bar{m}$ readings in #1409 appear monotonically decreasing and so regular? Because those five points all share the same configuration—differences concentrate between the primary category and the secondary category, while margins simultaneously shift from flat to sharp. Monotonic decrease is the product of “fixed configuration, sweeping margins,” not an intrinsic law of $C$. Once the configuration changes, $C$ can even reverse under the same $p_e$. This explains the anomaly in the low-consistency region: 0.60 and 0.75 lie on the same level, not due to measurement noise, but because two mechanisms cancel each other at this intersection—flatter margins cause per-category proportions to approach, increasing the number of non-zero components in the sign vector and raising the upper bound of variance; simultaneously, smaller differences narrow the true fluctuation.

VI. This Result is Subtractive for Practice

The implication of this result for practice is subtractive rather than additive. First, the threshold can be calculated before recruitment, but this requires reporting the complete expected margin distribution for each of the two readers, rather than just a single $p_e$ or a $\kappa$ target value—this aligns with methodological literature requirements: when solving for sample size based on target precision, the complete margin must be specified; providing only the coefficient cannot determine $n$. Second, any route of “estimating $C$ via a small pilot sample and extrapolating” is invalid, because $C$’s sensitivity to configuration is independent of $n$, and pilot studies cannot provide configuration information. Third, if practice allows only a single number, that number should take the upper bound based on the most unfavorable configuration—in this grid, configurations splitting across two categories generally yield higher $C$; using this as a conservative estimate will not underestimate the threshold.

Evidence and Limitations

Verified (pure algebra and Monte Carlo, reproducible): The linearized variance of TVD depends solely on the two readers’ margins; the sign vector and covariance matrix can be explicitly written; zero-difference categories must be supplemented with $0.25 \times (1 – 2/\pi) \times$ variance term via folded normal correction, otherwise naive linearization systematically underestimates; as the number of zero-difference categories increases from 0 to 3, the underestimation median rises from 0% to 12.4%, with a worst-case of 17%; the corrected closed form shows median relative error of 0.9% and maximum 2.5% against simulation across 54 grid points at scale of 50,000 cases.

Partially Verified: $C$’s sensitivity to “difference location” (spanning 1.14 to 1.32 times under same $p_e$ and TVD) was swept only across three-category configurations (far-end single, near-end single, split-between-two-categories), not exhaustively over the entire support set; the rule “two-category-split configuration yields higher $C$” holds consistently across nine grid points where $K=3$ to 6 and $p_e=0.40$ to 0.80, but cannot be proven for all configurations; thus it serves as empirical basis for conservative upper bound rather than a theorem.

To Be Verified: In real judgment studies, the distribution of “difference location” has never been systematically reported—published studies generally report only $\kappa$, AC1, and 95% intervals, rarely the complete margins for each of the two readers, making it impossible to infer configuration; this defines the applicability boundary of this conclusion and serves as the entry point for next work. Additionally, whether the closed form of $C$ remains valid when $K > 6$ or structural zero cells exist (categories never used by either reader) was not tested in this round.

L1 Network Status: The L1 edge table and corresponding network file lack the edge “variance structure of judgment consistency metrics.” The closest is the E-060 label premise (status conflict: PMID 30383398 reports inter-physician diagnostic consistency in open populations as low; authors call it “concerning”; KB-317/334 reports AC2 has run successfully in real clinical scenarios, pointing out the practical pitfall that “prevalence $\neq$ assessor-observed prevalence”; KB-333 reports that the direction of $\kappa$’s sensitivity to ordinal/nominal scales opposes the Gwet series). This edge only registers “whether label reliability is sufficient,” not “what determines the variance of reliability metrics.” Hence this paper caps $C$ and does not advocate any verified mechanism. This topic has not yet entered the L1 corresponding network; the gap is registered.

Further Reading

Evidence Level: C
References: PMID 11512137, PMID 22560852, PMID 36123642, PMID 38283651, PMID 23627889, PMID 25168681, PMID 15733050


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *