Evidence level: C
上一篇把所有结论压在一个从未测过的量上:两位判读者对「无变化」的占比差 δ。现在把它补上——但先要拆掉一个误会:δ 不是「判得准不准」的副产品,它是「两位读者心里那把尺子的刻度间距一样不一样」的直接读数。而这两种来源,量级差一个数量级。
正文
上一篇算清了偏倚 δ 如何压低 kappa 的天花板:m=0.90 时 δ=1pp 只掉 5%,δ=5pp 掉 27%,m=0.95 时 δ=5pp 直接掉一半(PMID 8501467 指出 kappa 混着偏倚与流行度两个可分离指数)。但那张表扫的是 δ 从 0 到 0.10,而真实 δ 到底多大,全篇一个字都没有。一个决定「要不要改报告规范」的判据,压在一个没人测过的数上,这件事本身就不合格。
一、先纠一个执行层面的错:δ 不能在「每个人」身上算
按最初设想,取 60 名受试者、3 位判读者各独立判一次,然后「逐人算两两主档占比差 δᵢⱼ」。这一步做不出来,原因是算术上的:对单独一个人,每位读者只给一个档位,所以这个人的「主档占比」只能取 0 或 1。A 判「无变化」、B 判「改善」,这一人的 δ 就是 1;两人都判「无变化」,δ 就是 0。δ 只能取两个值,它不是偏倚的估计量,它就是「这一次有没有分歧」本身。
δ 的定义是边际分布之差,而边际分布只在整个样本上存在,它天然是一个标量,不是一个分布。想让它有分布,唯一合法途径是把样本切成若干独立批次(分层、多中心、不同判读者组),在每批上算一个 δ̂,再谈 δ̂ 的离散程度。这一点不澄清,后面所有「δ 的 95 分位」「δ 对 m 的回归」都会算成噪声。### 二、δ̂ 的精度:这是全部判据的射程上限
按配对设计,δ̂ 的抽样方差约等于两位读者主档判定不一致率 (b+c) 除以 n。取不一致率 0.10,95% 置信区间半宽是:
| n | 60 | 120 | 300 | 960 | 3840 |
|---|---|---|---|---|---|
| 半宽 | 8.0pp | 5.7pp | 3.6pp | 2.0pp | 1.0pp |
结论很硬:n=60 的样本,δ̂ 的置信区间有 16pp 宽,而判据要区分的是 1pp 与 5pp,差了两个数量级。要把半宽压到 2pp(能勉强分辨 1 与 5),需要约 960 人;压到 1pp 需要约 3840 人。
这意味着原判据「δ 的 95 分位 <1pp 就算良性区」在 n=60 上永远会给出「不合格」——哪怕真实 δ 严格为零,观测值的 95 分位也会被抽样噪声推到 5pp 附近(蒙特卡洛 n=60 实测 5.10pp)。这不是数据有问题,是把噪声当信号;样本量本就该按目标参数精度反解(PMID 22560852)。
顺带说,「三位读者两两 δ 相差 >3pp 就说明二元模型不足」这条判据同样受制:n=60 时假阳性率约 42–46%,n=120 时约 30%,可用门槛在 n≈960 以上。
三、δ 有多大,取决于读者错在「偏」还是错在「散」
这是本篇的核心。用潜变量阈值模型能直接算出来:设每个人有个真实的变化程度 x~N(0,1),两位读者各自在 x 轴上放两个切口,切口之间是「无变化」,左边「加重」、右边「改善」。机制 A:两位读者的切口整体平移了 τ。 也就是一个偏保守(更容易判「有变化」),一个偏宽松。以 m=0.90 对称切口算:
| τ(单位=SD) | 0.2 | 0.3 | 0.5 | 0.7 |
|---|---|---|---|---|
| δ(对称切口) | −0.68pp | −1.52pp | −4.21pp | −8.19pp |
| δ(偏斜切口) | +0.44pp | +0.16pp | −1.38pp | −4.23pp |
注意两件事。第一,对称切口下的 δ 恒为负——整体平移只会让主档变小,「δ>0 的天花板塌陷」在这个机制下根本不出现。第二,偏斜切口下 δ 只在前段小幅转正(峰值约 0.4pp),随后转负而 |δ| 单调增,到 τ≈1.0 才达 10.7pp。也就是说要把 |δ| 推到 5pp,τ 得大到 0.74 SD,此时一个正常人被判进主档的概率已变了二十多个百分点。这种幅度的读者差异,研究里一眼就能看出来,不需要靠 δ 去发现。
所以单靠「偏一点」,δ 在常规范围内基本落在 1pp 以内。
机制 B:两位读者的切口间距不一样(一人尺子松、一人尺子紧)。 让读者二的切口围绕中点缩放 κ_s 倍,其余不变:
| m | κ_s=1.05 | 1.10 | 1.20 | 1.30 | 1.50 |
|---|---|---|---|---|---|
| 0.75 | 2.29pp | 4.43pp | 8.25pp | 11.52pp | 16.56pp |
| 0.90 | 1.58pp | 2.96pp | 5.16pp | 6.75pp | 8.64pp |
| 0.95 | 1.04pp | 1.89pp | 3.13pp | 3.92pp | 4.67pp |
刻度间距只差 20%,δ 就到 5pp;差 30%,到 7–12pp。 这个机制的产出效率比「平移」高一个数量级。这就是答案的形状:δ 不是「判得准不准」的函数,是「两人尺子刻度是否同源」的函数。 两位读者都判得不错、也都偏一点点,δ 会很小;两位读者各自稳定、内部一致度很高,但一个把「轻微改善」的标准定得比另一个松,δ 就会很大。而后一种情况 kappa 完全发现不了——它只看对角线。
四、一个真实领域的参考点
乳腺密度 BI-RADS 判读有个说明问题的例子:4 个中心、每中心 3 位放射科医师、共 800 张片子,读者间加权一致性 AC2=0.86(95%CI 0.85–0.88),按常规档位属「几乎完美」;同一批数据做边际同质性检验,各中心密度分布差异 P<0.001(PMID 37127220)。
高一致性可以与显著边际异质性同时成立。 这句话推翻「kappa 高就说明两人看法一致」的朴素理解:他俩对每个人的判断大致对得上,但对「这批人整体有多少属于哪一档」的看法系统性不同,而这正是 δ 的定义。边际配置如何把 kappa 与 AC1 拉出可观差距,已有直接证据(PMID 23627889);不同边际形态下各系数的行为分化也做过系统比较(PMID 25168681)。
五、判据要改,改的是问法
原判据问「δ 是 1pp 还是 5pp」,据此决定三元组还是四元组。这个问题问错了——它把一个由「机制」决定的量,当成了固定属性。
改法:先测尺度一致性,再谈 δ 的量级。 让每位读者对同一批人的「变化幅度」做一次连续打分(不分类,直接给 0–100),看两位读者打分的标准差之比。这个比值就是 κ_s 的直接估计,不依赖分类切口、不受主档占比影响,精度也远高于 δ̂(连续量的方差估计效率比比例差高得多)。比值在 0.9–1.1 之间,δ 结构上就在 1pp 量级;超过 1.2,δ 稳态就在 5pp 以上,四元组是硬要求。
δ 本身则只当事后核对量,不当研究目标。 研究做完报出 (m, δ, p₀, κ) 四元组,拿 δ 去查天花板——这是它该在的位置。把它当成「必须先测准的前置参数」,是把二阶量当一阶量。
证据与局限
已验证:δ 在样本层面定义、无法逐人计算(定义直接推得);δ̂ 的抽样标准误随 n 的 √n 收缩(配对比例差的标准结论);对称切口下 δ 恒非正(模型解析结果);κ_s 对 δ 的杠杆效应比 τ 高一个数量级(模型解析结果);高一致性可与显著边际异质性并存(PMID 37127220 直接报告)。
部分验证:潜变量阈值模型是三档有序裁决的标准心理测量框架,但它假设读者的判断只受单一潜在维度驱动,真实判读里可能混入「这个人我上次见过」之类的记忆效应,会额外制造边际差异,本模型不给这一项。δ 的真实量级在气功外气三档裁决里没有任何实测,本篇所有数字都是模型推演,不是实测值。
待验证:κ_s(读者打分标准差之比)这个代理量到底能不能预测 δ,需要在同一批数据上同时测两者;若相关系数够高(r>0.7),它可以替代 δ 当报告项。另需验证:把尺度一致性纳入训练(给出锚定样例)能否把 κ_s 压到 1.1 以内。
证据等级: C
参考文献: PMID 8501467, PMID 22560852, PMID 23627889, PMID 25168681, PMID 37127220
相关阅读
L1 网络状态:EDGE_TABLE 47 个概念中无「判读者」「一致性」「kappa」条目;CORRESPONDENCE_NETWORK 节点表三域(易学/物理/生理)均无「判读」节点,观察者侧仅有「气感报告|结构化自评」与「体质分型|量表」,
L2 以上边数 | 0。本文主题尚未进入 L1 对应网络。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
How Big Is the δ Skipped by the Entire Derivation: The Answer Lies Not in \”Zero Point Percentages\” but Whether the Reader’s Threshold Ruler Matches the Scale
Evidence level: C
In the previous article, all conclusions were compressed onto a single quantity that had never been measured: the difference δ between two raters’ proportions of cases assigned to the “no change” category. Now we supply this missing piece—but first, we must dismantle a misconception: δ is not merely a byproduct of how accurately the raters perform; it is the direct reading of whether the scale intervals in the two readers’ minds are identical. These two sources differ by an order of magnitude.
Main Text
The previous calculation clarified how bias δ suppresses the ceiling for kappa: when m=0.90, δ=1pp causes only a 5% drop, whereas δ=5pp causes a 27% drop; when m=0.95, δ=5pp directly halves the value (PMID 8501467 notes that kappa mixes two separable indices: bias and prevalence). Yet that table swept δ from 0 to 0.10, while the entire article contained not a single word about the actual magnitude of δ. A criterion intended to decide whether to revise reporting standards rests on a number that was never measured. This is fundamentally inadequate.
I. First Correction: δ Cannot Be Computed for “Each Individual”
According to the initial design, we would take 60 subjects and have each of three raters independently score them, then “compute δᵢⱼ pairwise for each individual.” This step is arithmetically impossible: for any single subject, each rater provides only one category, so the subject’s “proportion in the main category” can only be 0 or 1. If Rater A assigns “no change” and Rater B assigns “improvement,” then δ for that subject equals 1; if both assign “no change,” δ equals 0. δ can take only two values; it is not an estimator of bias—it is simply the indicator of whether disagreement occurred in this instance.
The definition of δ is the difference between marginal distributions, and marginal distributions exist only over the entire sample. Consequently, δ is naturally a scalar, not a distribution. To endow it with a distribution, the only legitimate approach is to partition the sample into independent batches (stratified, multi-center, or grouped by rater), compute an estimate δ̂ for each batch, and then discuss the variability of δ̂. Without clarifying this point, all subsequent claims—”the 95th percentile of δ” or “regression of δ on m”—will merely yield noise.
II. Precision of δ̂: This Sets the Upper Limit of All Criteria
Under a paired design, the sampling variance of δ̂ is approximately equal to the disagreement rate (b+c) divided by n. Taking a disagreement rate of 0.10, the half-width of the 95% confidence interval is:
| n | 60 | 120 | 300 | 960 | 3840 |
|---|---|---|---|---|---|
| Half-width | 8.0pp | 5.7pp | 3.6pp | 2.0pp | 1.0pp |
The conclusion is stark: with n=60, the confidence interval for δ̂ spans 16pp, yet the criterion must distinguish between 1pp and 5pp—a difference of two orders of magnitude. To shrink the half-width to 2pp (just enough to separate 1 and 5), we need approximately 960 subjects; to reach 1pp, we need about 3,840.
This means the original criterion—”δ’s 95th percentile <1pp indicates the benign region”—will always yield “unqualified” when n=60, even if the true δ is strictly zero. The 95th percentile of observed values will be pushed near 5pp by sampling noise (Monte Carlo simulation with n=60 yields 5.10pp). This is not a data problem; it is mistaking noise for signal. Sample size must be derived inversely from the target parameter precision (PMID 22560852).
Incidentally, the criterion “δ differences >3pp between pairwise comparisons of three raters indicates insufficient binary modeling” is equally constrained: with n=60, the false-positive rate is approximately 42–46%; with n=120, it drops to about 30%; the usable threshold requires n≈960 or more.
III. How Large Is δ? It Depends on Whether Readers Err Due to “Bias” or “Scatter”
This is the core of this article. Using latent-variable threshold models, we can compute it directly: suppose each subject has a true change magnitude x~N(0,1), and each rater places two cutpoints on the x-axis; the interval between them corresponds to “no change,” with the left tail representing “worsening” and the right tail “improvement.” Mechanism A: the two readers’ cutpoints are shifted globally by τ. That is, one reader is more conservative (more likely to assign “change”), while the other is more liberal. Using symmetric cutpoints with m=0.90:
| τ (units = SD) | 0.2 | 0.3 | 0.5 | 0.7 |
|---|---|---|---|---|
| δ (symmetric cutpoints) | −0.68pp | −1.52pp | −4.21pp | −8.19pp |
| δ (asymmetric cutpoints) | +0.44pp | +0.16pp | −1.38pp | −4.23pp |
Two observations matter. First, under symmetric cutpoints, δ is always negative—global shifting merely reduces the proportion in the main category; “δ>0 collapsing” never occurs under this mechanism. Second, under asymmetric cutpoints, δ becomes slightly positive only in the early stage (peaking at about 0.4pp), then turns negative with |δ| monotonically increasing, reaching 10.7pp only when τ≈1.0. Thus, to push |δ| to 5pp, τ must be as large as 0.74 SD, at which point the probability of a normal subject being classified into the main category shifts by more than twenty percentage points. Such magnitude of reader difference is immediately visible in studies; no need to rely on δ to detect it.
Therefore, relying solely on “a little bias,” δ remains within 1pp under conventional conditions.
Mechanism B: the two readers’ cutpoint intervals differ (one reader’s ruler is loose, the other tight). Let Rater 2’s cutpoints scale around the midpoint by a factor κ_s, keeping everything else unchanged:
| m | κ_s=1.05 | 1.10 | 1.20 | 1.30 | 1.50 |
|---|---|---|---|---|---|
| 0.75 | 2.29pp | 4.43pp | 8.25pp | 11.52pp | 16.56pp |
| 0.90 | 1.58pp | 2.96pp | 5.16pp | 6.75pp | 8.64pp |
| 0.95 | 1.04pp | 1.89pp | 3.13pp | 3.92pp | 4.67pp |
A mere 20% difference in scale intervals yields δ≈5pp; a 30% difference pushes δ to 7–12pp. This mechanism’s output efficiency is an order of magnitude higher than “shifting.” This is the shape of the answer: δ is not a function of how accurately the raters judge; it is a function of whether the two readers’ rulers share the same scale. If both raters perform well and are slightly biased, δ remains small; if both raters are internally consistent but one sets the standard for “slight improvement” more loosely than the other, δ becomes large. In the latter case, kappa cannot detect it at all—it only examines the diagonal.
IV. A Reference Point from a Real Domain
Breast density BI-RADS interpretation offers an illustrative example: four centers, three radiologists per center, totaling 800 images; the weighted inter-rater agreement AC2=0.86 (95%CI 0.85–0.88), which conventionally belongs to the “almost perfect” category; yet marginal homogeneity testing on the same data shows significant differences in density distribution across centers, P<0.001 (PMID 37127220).
High agreement can coexist with significant marginal heterogeneity. This overturns the naive understanding that “high kappa means the raters agree on everyone”: they may roughly match on individual cases, but their systematic views on “what proportion of the whole sample belongs to each category” differ substantially—and this is precisely δ’s definition. How marginal configuration pulls kappa apart from AC1 has direct evidence (PMID 23627889); systematic comparisons of coefficient behaviors under different marginal forms have also been conducted (PMID 25168681).
V. The Criterion Must Be Revised—by Changing the Question
The original criterion asks “Is δ 1pp or 5pp?” to decide between ternary and quaternary grouping. This question is flawed—it treats a quantity determined by “mechanism” as a fixed attribute.
The revision: first measure scale consistency, then discuss δ’s magnitude. Have each rater provide a continuous score of “change magnitude” for the same batch of subjects (no categorization, directly 0–100), and compute the ratio of standard deviations between the two raters’ scores. This ratio is the direct estimate of κ_s, independent of categorical cutpoints and unaffected by main-category proportions, with precision far superior to δ̂ (variance estimation of continuous quantities is much more efficient than proportion differences). If the ratio lies between 0.9–1.1, δ structurally falls within 1pp; exceeding 1.2, δ’s steady state is above 5pp, making quaternary grouping mandatory.
δ itself should serve only as a post-hoc check, not the research target. After completing the study and reporting the quadruple (m, δ, p₀, κ), use δ to verify the ceiling—that is where it belongs. Treating it as “a prerequisite parameter that must be measured accurately first” mistakes second-order quantities for first-order ones.
Evidence and Limitations
Verified: δ is defined at the sample level and cannot be computed per individual (directly implied by definition); the standard error of δ̂ shrinks with √n (standard result for paired proportion differences); under symmetric cutpoints, δ is always non-positive (model analytic result); κ_s exerts an order-of-magnitude greater leverage on δ than τ (model analytic result); high agreement can coexist with significant marginal heterogeneity (PMID 37127220 directly reports this).
Partially verified: The latent-variable threshold model is the standard psychometric framework for three-tier ordered adjudication, but it assumes readers’ judgments are driven solely by a single latent dimension; real adjudication may mix memory effects like “I’ve seen this person before,” creating additional marginal differences not captured by this model. The true magnitude of δ has no direct measurement in qigong external qi three-tier adjudication; all numbers in this article are model-derived, not measured.
To be verified: whether κ_s (the ratio of readers’ scoring standard deviations) can predict δ must be tested by measuring both simultaneously on the same dataset; if the correlation coefficient is sufficiently high (r>0.7), it can replace δ as the reported metric. Additionally, we must verify whether incorporating scale consistency into training (providing anchored examples) can suppress κ_s below 1.1.
Evidence Level: C
References: PMID 8501467, PMID 22560852, PMID 23627889, PMID 25168681, PMID 37127220
Related Reading
- Two raters’ proportion difference of five points widens the ceiling, not the bandwidth
- When the main category accounts for 90%, how high can kappa go: the closed interval dominated by Σp²
- How much bandwidth difference is required before comparability fails: inverse solution of the comparability threshold
- Resonance consciousness field research pool
L1 Network Status: Among the 47 concepts in EDGE_TABLE, there are no entries for “raters,” “agreement,” or “kappa”; the CORRESPONDENCE_NETWORK node table lacks “adjudication” nodes across all three domains (Yijing Studies, physics, physiology); the observer side contains only “qi sensation report | structured self-assessment” and “constitution typing | scale,” with
L2+ edges | 0. This article’s topic has not yet entered the corresponding L1 network.
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.