谁的尺子歪了,这个问题数学上没有答案:一个恒亏一秩的模型,加多少判读者都补不上

Written by

in

Language

Evidence level: C

上一篇证明训练只练尺度、不练零点。这一篇往更前提退一步:「零点歪了多少」这个量,在只有打分、没有刻度的条件下,根本不是估不准,而是估不出来。但同一份数据里,两人零点之差是能估的——这两件事常被混为一谈,代价是把一整类研究判成不可做。

正文

上一篇(#1402)的结论是:两位判读者的差异有两个自由度,尺度 a 和位置 b,而训练材料只压得住尺度、压不住零点。

顺着往下,会撞上一个更基础的问题:那我们把 b 测出来不就行了?

答案分两半,而且这两半方向相反。b 的绝对值测不出来,b 的差测得很准。 现有方法学讨论经常把这两半当成一件事,于是要么过度悲观(认为整条偏倚归因不可做),要么过度乐观(认为多加几位读者就解决了)。

一、加读者不降秩:这是恒亏一秩,不是样本量问题

读者模型是 s = a·x + b + ε。只看均值层面,R 位读者给出 R 个可观测的均值,未知量是 R+1 个:这批人的真实均值 μ,加上 R 个读者各自的零点 b₁…b_R。

R 位读者 → R 个观测,R+1 个未知
设计矩阵秩 = R,零空间维数 = 1(对任意 R)

实测 R 从 2 加到 8,亏的秩一直是 1,一点也不缩小:

设计 观测数 未知数 设计矩阵秩 亏秩
2 位读者 2 3 2 1
3 位读者 3 4 3 1
5 位读者 5 6 5 1
8 位读者 8 9 8 1
2 位读者 + 1 位金标准判读者 3 4 3 1
4 位读者 + 2 位金标准判读者 6 7 6 1

最后一行的意思是:再加一位「资深专家」当金标准也没用。资深专家也是一个读者,也带一个任意零点,他只是把 R 变成 R+1,亏的秩还是 1。这一步很容易被想岔——文献里常见的做法是「找一位权威来定标准」,而权威的标准答案本身也是从某个未标定的零点出发的。

要真正降秩,必须有一个有刻度的外部锚:不是「某位专家怎么判」,而是一个自带单位的物理量(比如皮温变化的摄氏度数、握力的牛顿值)。锚一进来,μ 被钉住,R+1 个未知里少一个,秩才补齐。

二、三个「基准」给出的不是三个答案,是同一个答案的三种写法

上一轮预设的判据是:三种基准算出的 τ̂ 若符号相反或幅度差超过 0.15 个标准差,就判「不可识别」。

实测发现这个判据测的不是识别性。同一份数据(真值 b₁=0.30、b₂=0.05),三种基准拟合出的残差平方和逐位相同,差值是 0.0:

基准假设 b̂₁ b̂₂ τ̂ = b̂₂−b̂₁ 残差平方和
认定 μ=0.00 +0.300 +0.050 −0.2500 330.806977
认定 μ=0.35 −0.050 −0.300 −0.2500 330.806977
认定 μ=−0.20 +0.500 +0.250 −0.2500 330.806977

三行的 τ̂ 一模一样。所谓「符号相反」,是读者一为基准与读者二为基准的差别(τ₁₂ = −τ₂₁),这是定义的方向在这个翻转,不是估计值在打架;所谓「幅度差 50%」,是「均值去中心」把差取了一半(报的是半差),也是定义。

所以这三个数不是三个互相矛盾的估计,是同一个可识别量在三种约定下的三种写法。危险不在数学,在报告环节:同一份数据可以报出 +0.25、−0.25、−0.125 三个数,读者无从判断哪个是「真的」。规范该管的是「必须声明 τ 是谁减谁、有没有除 2」,而不是「τ 不可报」。

三、错的是绝对值,对的永远是差

绝对值那一侧确实彻底不可识别,而且无界。把 μ 分别当成 −5、0、+5,观测到的读者一均值是 0.285 时,b̂₁ 依次是 +5.285、+0.285、−4.715——μ 挪多少,b̂₁ 就朝反方向挪多少,一个都不少。

真实研究里 μ 没人知道,习惯做法是默认 μ=0(把样本均值当零点)。这个默认的代价可以量出来:真 μ=0.35、真 b₁=0.30、真 b₂=0.05 时,按 μ=0 解释得到 b̂₁=0.647(高估 116%)、b̂₂=0.395(高估 690%)。绝对偏移量错得一塌糊涂。

但 δ̂ = b̂₂ − b̂₁ = −0.252,真值 −0.25,n=120 与 n=480 都一样准。这就是本篇最实用的一句:偏倚的绝对值不可答,偏倚之差可答,而报告和研究需要的恰恰是后者。

四、n=120 的区间半宽刚好等于一个整 5 个百分点

差可估,但要算多准。真 τ = 0.25 个标准差时,τ̂ 的抽样精度(800 次重抽):

n SD(τ̂) 95% 区间半宽 折成「5pp 当量」的倍数
60 0.089 0.175 1.38
120 0.066 0.128 1.02
240 0.045 0.089 0.71
480 0.033 0.065 0.51
960 0.023 0.045 0.36

「5pp 当量」指位置差达到 0.126 个总标准差时,边际差正好 5 个百分点(实测 5.00 pp)。上一轮设计的 ≥120 例,其 95% 区间半宽是 0.128——整整一个 5pp 当量。也就是说 n=120 时,「这个位置差超过 5pp 了吗」这个问题,区间的下半段说没超、上半段说超了,判决力接近于零。要到 n=240 才降到 0.71 个当量、n=480 才 0.51。结论:要判 5pp 级的位置差,n≥240 是底线,480 才算稳。

顺带一个坑:那个 0.126 的门槛没有声明分母是哪个 σ。用总标准差算,b=0.141 给 5.00 pp;用噪声标准差算,b=0.063 只给 2.26 pp。差 2.2 倍,全看分母怎么取——凡引用这个门槛的地方,必须写明「相对总离散度」还是「相对测量噪声」。

五、不可识别不等于无所谓:校错方向比不校更糟

「谁偏了」答不出来,很容易被理解成「那就算了,别管」。这个理解是错的。三个动作的实测后果(真差 b₁=0.141、b₂=0):

  • 不校正、只如实报告两人不一致:δ = 5.02 pp
  • 校正对方向(把读者一减去 0.141):δ = 0.01 pp
  • 校正错方向(把读者二再加上 0.141):δ = 10.01 pp

校错方向把偏倚放大到两倍,比什么都不做还糟一倍。而可识别性告诉我们:在没有外部锚的情况下,「往哪个方向校」这个选择没有任何数据依据——它只能靠约定(比如「一律以资深者为准」)来定。这就把风险从「可能偏」变成了「有一半概率把偏倚翻倍」。

这才是可识别性缺失的真正代价:它不是让研究变得不可做,而是让任何主动校正动作都带一个 50% 的翻倍风险。对应的规范应当是:无外部锚时,只报差值、不报归属、不做方向性校正。

证据与局限

已验证: 秩亏恒为 1,与读者数无关(R=2…8 及含金标准的 3、6 观测设计,设计矩阵秩恒等于观测数,零空间维数恒为 1);三基准下残差平方和精确相等(差 0.0)、τ̂ 数值恒定;b̂ 绝对值随 μ 任意平移而无界(μ 取 −5 至 +5,b̂₁ 从 +5.285 到 −4.715);δ̂ = b̂₂−b̂₁ 在 μ 误设下基本无偏(−0.252 对真值 −0.25);τ̂ 的 95% 区间半宽 n=60/120/240/480/960 分别为 0.175/0.128/0.089/0.065/0.045 个标准差;校错方向使 δ 从 5.02 pp 变为 10.01 pp。

部分验证: 「加外部锚可降秩」是线性代数的直接推论,但真实研究中「皮温变化幅度」这类锚本身也带测量误差,锚不精确时秩是补上了、估计量的方差却会变大,这个折中本文没有算。0.126 这个 5pp 当量门槛来自上一篇的数值解,本文只复核了它在两种 σ 定义下的取值,未重新推导。

待验证: ① 三位以上读者 + 外部锚的完整设计中,锚的误差如何传播到 τ̂ 的方差,从未算过;② 真实判读数据里「一律以资深者为准」这条约定造成的方向错误率是多少,没有实测(本文只算了「错一次」的后果,没算「多久错一次」);③ 当 μ 本身随批次变化(不同批受试者人群构成不同)时,跨批次合并 τ̂ 是否仍然无偏,未检验。

相关阅读

证据等级: C
参考文献: PMID 2868172, PMID 8501467, PMID 22129533, PMID 39854081, PMID 31444807
L1 网络状态:CORRESPONDENCE_NETWORK.md 全表 grep「判读 / kappa / 一致性 / 信度 / 评分 / 读者 / 校准 / 偏倚 / 金标准 / 参照 / 基准 / 可识别 / 归属 / 真值」仅命中第 192 行「生克一致性 | 相生/相克方向能否复现 | 未测」(指五行生克,与判读信度无关);物理侧最近位为第 57 行「相位 / 节律 | — | 周期性耦合的关键」,语义限于两振荡锁相;生理/体验侧九项节点无「判读信度」「位置偏倚」「可识别性」任何节点;边表止于 E-050「六十甲子纳音 | 对应 | 60 个频率」(L1 / open,现代证据栏原文「尚无任何可测来源」);网络对齐指标行原文「L2 以上边数 | 0」。该主题尚未进入 L1 对应网络,已登记至 science-yixue-l1-gaps.md(「测量一致性」簇第 16 条)。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Whose ruler is crooked, a question with no mathematical answer: adding any number of classifiers to a model that is one rank deficient cannot compensate

Evidence level: C

The previous proof showed that training only calibrates the scale, not the zero point. This paper steps back further: the magnitude of “how much the zero is bent” cannot be estimated at all under conditions where only scores are given, not scales. Yet within the same dataset, the difference between two zero points can be estimated. These two facts are often conflated, with the cost of declaring an entire class of studies infeasible.

Body

The conclusion of the previous paper (#1402) is that the difference between two raters involves two degrees of freedom: scale $a$ and position $b$, while the training material constrains only the scale, not the zero point.

Following this, we encounter a more fundamental question: Can’t we just measure $b$?

The answer splits into two halves, which point in opposite directions. The absolute value of $b$ cannot be measured, but the difference between $b$ values can be measured very accurately. Current methodological discussions often treat these two halves as one thing, leading either to excessive pessimism (that entire classes of bias attribution are infeasible) or excessive optimism (that adding more raters solves the problem).

1. Adding Raters Does Not Reduce Rank: This Is a Permanent Rank Deficiency of One, Not a Sample Size Issue

The rater model is $s = a \cdot x + b + \varepsilon$. Looking only at the mean level, $R$ raters provide $R$ observable means. The unknown quantities are $R+1$: the true mean $\mu$ of these people, plus the $R$ individual zero points $b_1 \dots b_R$.

R raters → R observations, R+1 unknowns
Design matrix rank = R, null space dimension = 1 (for any R)

Empirically, increasing $R$ from 2 to 8 leaves the missing rank always at 1, never shrinking:

Design Observations Unknowns Design Matrix Rank Deficient Rank
2 raters 2 3 2 1
3 raters 3 4 3 1
5 raters 5 6 5 1
8 raters 8 9 8 1
2 raters + 1 gold-standard rater 3 4 3 1
4 raters + 2 gold-standard raters 6 7 6 1

The last row means: adding another “senior expert” as a gold standard is useless. A senior expert is also just a rater, carrying an arbitrary zero point; they merely increase $R$ to $R+1$, leaving the deficient rank still at 1. This step is easily misunderstood—common literature practice is “find an authority to set the standard,” yet the authority’s standard answer itself originates from some unstabilized zero point.

To truly reduce the rank, we need an external anchor with units: not “how a certain expert judges,” but a physical quantity carrying units (e.g., skin temperature change in degrees Celsius, grip strength in Newtons). Once this anchor enters, $\mu$ is pinned down, reducing the $R+1$ unknowns by one, and the rank is completed.

2. Three “Baselines” Do Not Give Three Answers, They Give Three Ways of Writing the Same Answer

The previous assumption stated that if the estimated $\hat{\tau}$ from three baselines have opposite signs or a magnitude difference exceeding 0.15 standard deviations, it should be judged “unidentifiable.”

Empirical testing shows this criterion does not measure identifiability. For the same dataset (true values $b_1=0.30, b_2=0.05$), the residual sum of squares fitted by three baselines is identical at each position, with a difference of 0.0:

Baseline Assumption $\hat{b}_1$ $\hat{b}_2$ $\hat{\tau} = \hat{b}_2 – \hat{b}_1$ Residual Sum of Squares
Assume $\mu=0.00$ +0.300 +0.050 −0.2500 330.806977
Assume $\mu=0.35$ −0.050 −0.300 −0.2500 330.806977
Assume $\mu=-0.20$ +0.500 +0.250 −0.2500 330.806977

The $\hat{\tau}$ values are identical across all three rows. The so-called “opposite signs” refers to the difference between Rater 1 as baseline and Rater 2 as baseline ($\tau_{12} = -\tau_{21}$); this is a reversal of definition direction, not a conflict in estimates. The so-called “50% magnitude difference” refers to “mean-centering” halving the difference (reporting half-difference), which is also a matter of definition.

Thus, these three numbers are not three contradictory estimates; they are three ways of writing the same identifiable quantity under different conventions. The danger lies not in mathematics, but in reporting: the same dataset can report +0.25, −0.25, and −0.125, leaving readers unable to judge which is “true.” Standards should govern “must declare who minus whom, whether divided by 2,” not “τ cannot be reported.”

3. The Absolute Value Is Wrong, the Difference Is Always Right

The absolute value side is indeed completely unidentifiable and unbounded. Setting $\mu$ to −5, 0, or +5 respectively, when the observed mean of Rater 1 is 0.285, yields $\hat{b}_1$ values of +5.285, +0.285, and −4.715 respectively—as $\mu$ shifts, $\hat{b}_1$ shifts in the opposite direction by exactly the same amount.

In real research, $\mu$ is unknown; the habitual practice defaults to $\mu=0$ (treating the sample mean as the zero point). The cost of this default can be quantified: when true $\mu=0.35$, true $b_1=0.30$, and true $b_2=0.05$, interpreting with $\mu=0$ yields $\hat{b}_1=0.647$ (overestimated by 116%) and $\hat{b}_2=0.395$ (overestimated by 690%). The absolute offset is utterly wrong.

However, $\hat{\delta} = \hat{b}_2 – \hat{b}_1 = -0.252$, matching the true value of −0.25 with $n=120$ and $n=480$ equally accurate. This is the most practical sentence of this paper: the absolute value of bias cannot be answered, but the difference of biases can be answered, and what reporting and research need is precisely the latter.

4. The Half-Width of the Interval for $n=120$ Equals Exactly One Full 5 Percentage Points

The difference can be estimated, but we must calculate how accurately. When the true $\tau = 0.25$ standard deviations, the sampling precision of $\hat{\tau}$ (800 resamples):

n SD($\hat{\tau}$) 95% Interval Half-Width Multiplier of “5pp Equivalent”
60 0.089 0.175 1.38
120 0.066 0.128 1.02
240 0.045 0.089 0.71
480 0.033 0.065 0.51
960 0.023 0.045 0.36

“5pp equivalent” refers to when the positional difference reaches 0.126 of the total standard deviation, the marginal difference is exactly 5 percentage points (measured as 5.00 pp). The previous design’s $\ge 120$ cases have a 95% interval half-width of 0.128—exactly one 5pp equivalent. That means at $n=120$, the question “does this positional difference exceed 5pp?” has the lower half of the interval saying no, and the upper half saying yes, making the decision power nearly zero. Only at $n=240$ does it drop to 0.71 equivalents, and at $n=480$ to 0.51. Conclusion: to judge positional differences at the 5pp level, $n \ge 240$ is the baseline, and $n=480$ is considered stable.

A side pitfall: the 0.126 threshold did not declare which $\sigma$ is in the denominator. Using total standard deviation, $b=0.141$ gives 5.00 pp; using noise standard deviation, $b=0.063$ gives only 2.26 pp. A difference of 2.2 times, entirely depending on how the denominator is chosen—any reference to this threshold must specify “relative to total dispersion” or “relative to measurement noise.”

5. Unidentifiable Does Not Mean Irrelevant: Correcting in the Wrong Direction Is Worse Than Not Correcting

That “who is biased” cannot be answered is easily understood as “then forget it, don’t worry.” This understanding is wrong. The empirical consequences of three actions (true difference $b_1=0.141, b_2=0$):

  • No correction, only reporting the inconsistency between two raters: $\delta = \mathbf{5.02\ pp}$
  • Correcting in the right direction (subtracting 0.141 from Rater 1): $\delta = \mathbf{0.01\ pp}$
  • Correcting in the wrong direction (adding 0.141 to Rater 2): $\delta = \mathbf{10.01\ pp}$

Correcting in the wrong direction doubles the bias, making it twice as bad as doing nothing. Identifiability tells us: without an external anchor, the choice of “which direction to correct” has no data basis—it can only rely on convention (e.g., “always use the senior expert”). This turns the risk from “possibly biased” into “50% probability of doubling the bias.”

This is the true cost of missing identifiability: it does not make research infeasible, but makes any active correction carry a 50% doubling risk. The corresponding standard should be: without an external anchor, report only differences, do not report attribution, and do not perform directional correction.

Evidence and Limitations

Verified: Rank deficiency is permanently 1, independent of the number of raters ($R=2 \dots 8$ and gold-standard designs with 3, 6 observations; the design matrix rank always equals the number of observations, and the null space dimension is always 1); residual sum of squares under three baselines are exactly equal (difference 0.0), and $\hat{\tau}$ values remain constant; absolute values of $\hat{b}$ shift unboundedly as $\mu$ arbitrarily translates (with $\mu$ ranging from −5 to +5, $\hat{b}_1$ ranges from +5.285 to −4.715); $\hat{\delta} = \hat{b}_2 – \hat{b}_1$ is essentially unbiased under mis-specification of $\mu$ (−0.252 vs true −0.25); 95% interval half-widths for $\hat{\tau}$ at $n=60/120/240/480/960$ are 0.175/0.128/0.089/0.065/0.045 standard deviations respectively; correcting in the wrong direction changes $\delta$ from 5.02 pp to 10.01 pp.

Partially Verified: “Adding an external anchor reduces rank” is a direct consequence of linear algebra, but in real research, anchors like “skin temperature change amplitude” themselves carry measurement error; when the anchor is imprecise, the rank is filled but the variance of the estimator increases, a trade-off not calculated in this paper. The 0.126 threshold for the 5pp equivalent comes from numerical solutions in the previous paper; this paper only verified its values under two $\sigma$ definitions, without re-deriving it.

To Be Verified: ① In complete designs with three or more raters plus an external anchor, how the anchor’s error propagates to the variance of $\hat{\tau}$ has never been calculated; ② In real rating data, what is the rate of directional error caused by the convention “always use the senior expert,” has not been empirically tested (this paper only calculated the consequence of “one wrong correction,” not “how often it goes wrong”); ③ When $\mu$ itself varies across batches (different batches have different subject population compositions), whether merging $\hat{\tau}$ across batches remains unbiased has not been tested.

Related Reading

Evidence Level: C
References: PMID 2868172, PMID 8501467, PMID 22129533, PMID 39854081, PMID 31444807
L1 Network Status: CORRESPONDENCE_NETWORK.md full table grep for “rater / kappa / consistency / reliability / scoring / reader / calibration / bias / gold standard / reference / baseline / identifiable / attribution / true value” only hits row 192 “Sheng Ke Consistency | Can Sheng/Xiang directions be reproduced | Not tested” (referring to Five Elements Sheng/King, unrelated to rater reliability); the physical side’s latest position is row 57 “Phase / Rhythm | — | Key of periodic coupling,” with semantics limited to two oscillators phase-locked; physiological/experience side has nine nodes but none for “rater reliability,” “positional bias,” or “identifiability”; the edge table stops at E-050 “Sixty Jiazi Yin-Yin | Correspondence | 60 frequencies” (L1 / open, original text in modern evidence column states “no measurable source yet”); network alignment indicator row originally “L2 and above edges | 0”. This topic has not yet entered the L1 corresponding network, and is registered in science-yixue-l1-gaps.md (cluster “measurement consistency,” item 16).


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *