标准差之比这把尺子,量不出尺子挪了地方:一个精确等于 1.000 的合格读数,背后可以是八个百分点的偏倚

Written by

in

Language

Evidence level: C

上一篇建议用「两位判读者连续打分的标准差之比」来替代那个从没测过的偏倚 δ。这个建议只有一个漏洞,但它致命:δ 由两个自由度决定,而这个比值只读其中一个。漏掉的那个自由度,能把 δ 直接推到八个点,同时让这个比值精确等于 1.000。

正文

上一篇把偏倚 δ 的量级归给「两位读者心里的尺子刻度是否同源」,并提出一个改法:让每位读者对同一批人的变化幅度做 0–100 连续打分,看两人打分的标准差之比 κ_s;比值在 0.9–1.1 就是尺子同源、δ 结构上在 1pp 量级。理由是连续量的方差估计效率远高于比例差,精度能高一个数量级。

这个改法错在两处:漏了一个自由度;「效率高一个数量级」这句话,方向是反的。

一、κ_s 对位置是精确盲的

每位读者对同一个人打分 s_j = a_j·x + b_j,x 是真实变化程度。于是读者的全部行为只有两个自由度:尺度 a(尺子多长)和位置 b(零点在哪)。κ_s = a₁/a₂ 只读第一个。

现在做纯平移:a₁ = a₂ = 1,b₁ = 0,b₂ = τ。此时第二位打分恒等于第一位加常数,两人打分的标准差完全相同,κ_s 精确等于 1.000——不是「约等于」,是恒等式。但主档占比照样分家:

τ(单位=共同潜变量 SD) 0.2 0.4 0.6 0.9
κ_s 1.000 1.000 1.000 1.000
δ(m=0.75) 0.9pp 3.7pp 8.1pp 17.1pp
δ(m=0.90) 0.7pp 2.7pp 6.0pp 13.4pp

(δ 定义为「读者一主档占比 − 读者二」,正值表示读者二更倾向把人有变化、主档更少。)

要把 δ 推到 5pp 这条硬线,纯平移只需 τ = 0.47 SD(m=0.75)或 0.55 SD(m=0.90)。0.47 个标准差有多大:m=0.75 时两条切口原本在 ±1.15 SD、下侧「判进加重」与上侧「判进改善」各占 12.5%,平移后落到 −0.68 与 1.62,下侧涨到 24.7%、上侧压到 5.3%。读者二那边「加重」的人数直接翻了一倍——这种差异把两张边际表并排一看就露,但 κ_s 报出来是 1.000,一个数都察觉不到。

二、门在两个方向上同时出错,错法随 m 变

反解「δ 不超过 5pp」允许的 κ_s 区间(纯尺度差):

m 0.60 0.75 0.85 0.90 0.93
允许下限 0.898 0.901 0.890 0.875 0.858
允许上限 1.110 1.114 1.143 1.192 1.284

低档位(m≤0.80)与名义 ±0.10 大体吻合,误差 5%–7%,改法看着没问题。但主档一集中,区间就朝两边撕开:m=0.90 到 [0.875, 1.192],m=0.93 到 [0.858, 1.284]。而真实三档裁决的主档占比普遍在 0.8–0.95。

以 m=0.90 为例,±0.10 同时干两件坏事:真 κ_s=1.15 的读者组,δ 只有 4.2pp、完全合格,却落在门外的「超界」侧;真 κ_s=0.88 的读者组,δ 已到 4.8pp、贴着硬线,反而被判合格。把「合法区」和「名义门」叠加,落在门外却仍合格的宽度占整个合法区:m=0.75 是 6.6%,m=0.85 是 20.7%,m=0.90 是 36.8%,m=0.93 是 53.0%。同一条线,在主档集中时三分之一到一半的合法配置会被误判,而没有任何固定数字能同时修好两边。

三、「效率高一个数量级」这句话,方向是反的

上一篇的推理混淆了两件事——κ_s 估得准,和δ 换算得准。中间隔着一个汇率。

δ 对 κ_s 的斜率在 κ_s=1 处是 22.9–48.3 pp/单位(m=0.70 最高 48.3,m=0.90 是 33.9,m=0.95 是 22.9)。要把 δ 的 95% 置信区间半宽压到 2pp,κ_s 的标准误得压到 0.0215;按 σ 之比的抽样分布反解,每读者需要约 2155 人。而 δ 直接测(假定不一致率 10%)只要约 960 人。贵 2.2 倍,不是便宜十倍。

只有 m≥0.92 才翻转(m=0.95 时 505 对 960,省一半)。小样本更糟:n=60 时走 κ_s 换回来的 δ 区间半宽是 12.3pp,直测是 8.0pp。

四、这道门在小样本上自己就过不去

真 κ_s 严格等于 1.0 时,估计值落进 [0.90, 1.10] 的概率:n=30 只有 40.7%,n=60 是 55.6%,n=120 是 72.4%,n=300 是 91.6%,要到 n≈388 才有 95%。一项 n=60 的研究,哪怕两人尺子一模一样,也有一半以上概率被判「尺度不合格」。

这和上一篇批判过的「n=60 时 δ 判据永远给出不合格」是同一个病,只换了个参数。实况更不乐观:用 kappa 的一致性研究样本量中位数只有 71(四分位 50–233),而 82 项一致性研究里只有 27 项(33%)给了样本量依据(PMID 36123642)。多数研究正好落在这道门最不可靠的位置,且没人算过自己该有多少人。样本量本就该按目标精度反解(PMID 22560852;改用 ICCa 同理,须按区间宽度反解,PMID 39285135)。

五、改法:把双向判据改成单向

κ_s 唯一站得住的是单向:κ_s 超界 ⇒ 必存在尺度差 ⇒ δ 必不为零(量级还得看 m)。反向不成立——κ_s 合格什么都不说明,因为位置差与它无关。

非要用 κ_s 当前置筛选,两件事必须同做:① 同报位置参数 τ̂(两人切口中点之差,单位=共同潜变量 SD),或干脆直接报 δ̂ 加置信区间;② 样本量按 δ 的目标精度反解后乘 2.2。τ̂ 的估法有个坑:把两人连续打分做回归,斜率给尺度比、截距给位置差,但两人的测量误差都会压低斜率,普通最小二乘会系统性低估尺度差,须用主轴回归或 Deming 回归。

这套处境在方法学上早有对应:固定偏倚(整体系统差)与比例偏倚(随量级变化的差)必须分开检验,只报一个平均差和一致性界限会把系统差藏起来(PMID 2868172)。连续打分的标准差之比读的正是「比例偏倚」,它天生看不见「固定偏倚」。Kendall 早就指出 kappa 混着偏倚与流行度两个可分离指数,任一单指标都可能漏检另一个维度(PMID 8501467)。高一致率与显著边际异质性可以同时成立,这个反例在真实放射判读数据里已被直接量到(PMID 37127220)。

证据与局限

已验证:κ_s 对纯平移精确不敏感(a₁=a₂ 时 σ₁/σ₂ 恒为 1,算术恒等);δ 是尺度与位置的二元函数(潜变量阈值模型解析结果);纯尺度门下 δ=5pp 的临界 κ_s 随 m 外扩,名义 ±0.10 在 m≥0.85 上误判率 20.7%–53.0%(数值反解);真值 1 时 κ_s 的估计量服从 √F 分布,n≈388 才有 95% 落入 ±0.10(F 分布解析);固定偏倚与比例偏倚须分开检验(PMID 2868172 直接论述)。

部分验证:潜变量阈值模型假设单一潜在维度、两位读者共用同一条 x 轴。真实判读可能混入记忆效应、病例顺序效应,这些既不是纯尺度也不是纯位置,会同时污染两个参数,本模型不给这一项。δ 直测的 n≈960 按不一致率 10% 算,真实三档裁决可能在 5%–30%,对应 n 从 480 到 2881。所有数字都是模型推演,气功外气三档裁决里没有任何实测的 κ_s 或 τ。

待验证:① 真实三档判读数据上 κ_s 与 δ 的相关系数到底多少——上一篇给的准入线是 r>0.7,本文只证了「纯平移机制下 r 可以接近 0」,没测真实数据。② τ̂ 用主轴回归估计的偏差与稳健性,尚无数值。③ 「锚定样例训练」压 κ_s 时能否同时压住 τ——这是两个自由度,只训一个可能压不住另一个。

证据等级: C
参考文献: PMID 2868172, PMID 22560852, PMID 37127220, PMID 39285135, PMID 8501467, PMID 36123642

相关阅读

L1 网络状态:CORRESPONDENCE_NETWORK.md 三域节点表中,物理侧仅「相位 / 节律 | — | 周期性耦合的关键」,生理侧九项节点(HRV / 皮温皮电 / 呼吸率 / EMG / EEG / 胃肠 / 气感报告 / 体质分型)无「判读者」「评分者」「连续打分」任何节点;全表检索「判读 / 信度 / 一致率 / 尺度 / 偏倚 / 代理量」零命中;EDGE_TABLE_science-yixue.md 47 个概念中无相关条目;网络对齐指标行原文「L2 以上边数 | 0」。该主题尚未进入 L1 对应网络。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

The ratio of standard deviations as a ruler cannot measure when the ruler has shifted: a qualifying reading precisely equal to 1.000 can hide an eight-percentage-point bias behi…

Evidence level: C

In the previous post, it was suggested that “the ratio of standard deviations between two raters scoring consecutively” should replace the bias δ, which was never measured. This suggestion has only one flaw, but it is fatal: δ is determined by two degrees of freedom, while this ratio reads only one. The missing degree of freedom can push δ directly to eight points, while keeping this ratio exactly equal to 1.000.

Main Text

The previous post attributed the magnitude of bias δ to “whether the scale markings in the minds of two readers share the same origin,” and proposed a modification: have each reader give continuous scores from 0–100 on the magnitude of change for the same batch of people, and examine the ratio of standard deviations κ_s of their scores; a ratio between 0.9 and 1.1 indicates the scales share the same origin, and δ is structurally on the order of 1 part per hundred (pp). The rationale is that the variance estimation efficiency of continuous quantities is far superior to that of proportional differences, allowing precision one order of magnitude higher.

This modification errs in two ways: it omits one degree of freedom; and the phrase “one order of magnitude more efficient” is reversed in direction.

I. κ_s is precisely blind to location

Each reader’s score for person $j$ is $s_j = a_j \cdot x + b_j$, where $x$ represents the true magnitude of change. Thus, a reader’s entire behavior involves only two degrees of freedom: scale $a$ (the length of the ruler) and location $b$ (where the zero point lies). The ratio $\kappa_s = a_1/a_2$ reads only the first.

Now consider pure translation: $a_1 = a_2 = 1$, $b_1 = 0$, $b_2 = \tau$. In this case, the second reader’s scores are always equal to the first plus a constant; the standard deviations of their scores are identical, so $\kappa_s$ is exactly 1.000—not “approximately,” but an identity. Yet the proportion of main records still diverges:

τ (units = common latent variable SD) 0.2 0.4 0.6 0.9
κ_s 1.000 1.000 1.000 1.000
δ (m=0.75) 0.9pp 3.7pp 8.1pp 17.1pp
δ (m=0.90) 0.7pp 2.7pp 6.0pp 13.4pp

(δ is defined as “reader one’s proportion of main records minus reader two”; positive values indicate reader two tends to mark people as having change, while main records are fewer.)

To push δ to the hard limit of 5pp, pure translation requires $\tau = 0.47$ SD (for $m=0.75$) or $0.55$ SD (for $m=0.90$). What is 0.47 standard deviations? When $m=0.75$, the two cut-offs originally lie at ±1.15 SD, with “判进加重” (判进加重) and “判进改善” (判进改善) each occupying 12.5% on the lower and upper sides respectively; after translation, they fall to −0.68 and 1.62, with the lower side rising to 24.7% and the upper side compressed to 5.3%. The number of people marked “加重” (加重) by reader two doubles directly—such differences become obvious when the two marginal tables are placed side-by-side, yet $\kappa_s$ reports 1.000, revealing nothing.

II. The gate errs in both directions, with error depending on m

Solving backwards for the allowable $\kappa_s$ interval under “δ not exceeding 5pp” (pure scale difference):

m 0.60 0.75 0.85 0.90 0.93
Allowable lower bound 0.898 0.901 0.890 0.875 0.858
Allowable upper bound 1.110 1.114 1.143 1.192 1.284

The low档位 (m ≤ 0.80) roughly matches the nominal ±0.10, with errors of 5%–7%, making the modification appear acceptable. But when main records concentrate, the interval tears apart on both sides: for $m=0.90$ it becomes [0.875, 1.192], and for $m=0.93$ it becomes [0.858, 1.284]. In real three-tier adjudication, the proportion of main records is generally between 0.8 and 0.95.

Taking $m=0.90$ as an example, the nominal ±0.10 simultaneously does two harmful things: a group with true $\kappa_s=1.15$ has δ only 4.2pp, fully qualified, yet falls on the “out-of-gate” side; a group with true $\kappa_s=0.88$ has δ already at 4.8pp, touching the hard limit, yet is judged qualified. Overlaying the “legal zone” and the “nominal gate,” the width that falls outside yet remains qualified accounts for: $m=0.75$ is 6.6%, $m=0.85$ is 20.7%, $m=0.90$ is 36.8%, and $m=0.93$ is 53.0%. Along the same line, when main records concentrate, one-third to half of all legally valid configurations are misjudged, and no fixed number can simultaneously fix both sides.

III. “One order of magnitude more efficient” is reversed in direction

The previous post’s reasoning conflates two matters—that $\kappa_s$ is estimated accurately, and that δ is converted accurately—separated by an exchange rate.

The slope of δ with respect to $\kappa_s$ at $\kappa_s=1$ ranges from 22.9 to 48.3 pp per unit (highest at $m=0.70$ with 48.3, $m=0.90$ yields 33.9, and $m=0.95$ gives 22.9). To compress the half-width of δ’s 95% confidence interval to 2pp, the standard error of $\kappa_s$ must be reduced to 0.0215; solving backwards via the sampling distribution of the ratio of σs, each reader requires approximately 2155 people. Measuring δ directly (assuming an inconsistency rate of 10%) requires only about 960 people. This is 2.2 times more expensive, not ten times cheaper.

Only when $m \geq 0.92$ does the direction flip (for $m=0.95$, 505 versus 960, saving half). Small samples fare worse: at $n=60$, the half-width of δ recovered via $\kappa_s$ is 12.3pp, whereas direct measurement yields 8.0pp.

IV. This gate cannot even pass itself with small samples

When the true $\kappa_s$ is strictly 1.0, the probability that the estimate falls into [0.90, 1.10] is: $n=30$ gives only 40.7%, $n=60$ yields 55.6%, $n=120$ reaches 72.4%, and $n=300$ achieves 91.6%; it takes $n \approx 388$ to reach 95%. A study with $n=60$, even if both rulers are identical, has over a 50% chance of being judged “scale-不合格”.

This matches the criticism leveled in the previous post that “at $n=60$, the δ criterion always yields failure,” differing only in parameter. Reality is even less optimistic: the median sample size for kappa consistency studies is 71 (interquartile range 50–233), and among 82 consistency studies, only 27 (33%) provided sample size justification (PMID 36123642). Most studies fall precisely where this gate is most unreliable, and no one has calculated how many subjects they should have. Sample size ought to be derived backwards from target precision (PMID 22560852; similarly for ICCa, it must be solved backwards via interval width, PMID 39285135).

V. Modification: Convert the bidirectional criterion into unidirectional

The only robust aspect of $\kappa_s$ is unidirectional: $\kappa_s$ out-of-bounds ⇒ there must be a scale difference ⇒ δ cannot be zero (the magnitude still depends on $m$). The reverse does not hold—$\kappa_s$ being qualified tells us nothing, because location difference is unrelated to it.

If we insist on using $\kappa_s$ as a pre-screen, two things must be done simultaneously: ① report the location parameter $\hat{\tau}$ (the difference between midpoints of the two cut-offs, units = common latent variable SD), or simply report $\hat{\delta}$ with its confidence interval; ② derive sample size by multiplying the target precision for δ by 2.2. Estimating $\hat{\tau}$ has a pitfall: regressing the two continuous scores gives slope as scale ratio and intercept as location difference, but measurement errors from both readers depress the slope; ordinary least squares systematically underestimates scale difference, requiring principal axis regression or Deming regression.

This situation has long counterparts in methodology: fixed bias (overall systematic difference) and proportional bias (difference varying with magnitude) must be tested separately; reporting only an average difference and consistency limits hides systematic bias (PMID 2868172). The ratio of standard deviations of continuous scores reads precisely “proportional bias,” which inherently cannot see “fixed bias.” Kendall long ago pointed out that kappa mixes two separable indices—bias and prevalence—and any single indicator may miss the other dimension (PMID 8501467). High consistency rates and significant marginal heterogeneity can coexist, a counterexample directly measured in real radiological reading data (PMID 37127220).

Evidence and Limitations

Verified: $\kappa_s$ is precisely insensitive to pure translation (when $a_1=a_2$, $\sigma_1/\sigma_2$ is always 1, an arithmetic identity); δ is a binary function of scale and location (analytical result from latent variable threshold model); under pure scale gates, the critical $\kappa_s$ for $\delta=5pp$ expands outward with $m$, and the nominal ±0.10 misjudgment rate reaches 20.7%–53.0% for $m \geq 0.85$ (numerical inversion); when the true value is 1, the estimator of $\kappa_s$ follows a $\sqrt{F}$ distribution, requiring $n \approx 388$ for 95% to fall within ±0.10 (analytical F distribution); fixed bias and proportional bias must be tested separately (PMID 2868172 explicitly discusses this).

Partially Verified: The latent variable threshold model assumes a single latent dimension and that two readers share the same x-axis. Real adjudication may mix memory effects, case-order effects, which are neither pure scale nor pure location, contaminating both parameters simultaneously; this model does not include such a term. Direct measurement of δ requires $n \approx 960$ assuming an inconsistency rate of 10%, while real three-tier adjudication may range from 5%–30%, corresponding to $n$ from 480 to 2881. All numbers are model-derived; there is no empirical $\kappa_s$ or $\tau$ in the three-tier adjudication of qigong external qi.

To Be Verified: ① What is the actual correlation coefficient between $\kappa_s$ and δ on real three-tier adjudication data—the准入线 (admission threshold) given in the previous post is $r > 0.7$, but this paper only proves that “under pure translation mechanism, $r$ can approach 0,” without measuring real data. ② The bias and robustness of $\hat{\tau}$ estimated via principal axis regression, lacking numerical values. ③ Whether “anchored sample training” suppressing $\kappa_s$ can simultaneously suppress $\hat{\tau}$—these are two degrees of freedom; training only one may not constrain the other.

Evidence Level: C
References: PMID 2868172, PMID 22560852, PMID 37127220, PMID 39285135, PMID 8501467, PMID 36123642

Further Reading

L1 Network Status: In the three-domain node table of CORRESPONDENCE_NETWORK.md, the physical side contains only “phase/rhythm | — | key of periodic coupling”; the physiological side has nine nodes (HRV / skin temperature/skin conductance / breathing rate / EMG / EEG / gastrointestinal / qi sensation report / constitution typing) but no nodes for “reader/rater/continuous scoring”; searching the entire table for “reading/reliability/consistency/scale/bias/proxy quantity” yields zero hits; EDGE_TABLE_science-yixue.md has 47 concepts but no relevant entries; the network alignment indicator row reads “L2 and above edge count | 0”. This topic has not yet entered the L1 corresponding network.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *