同一批数据、五个起点、五个答案、似然一模一样:位置漂移的可估性,比它的精度先垮

Written by

in

Language

Evidence level: C

上一篇算出了「位置漂移到多少会撞线」,这一篇要问:那个漂移量本身能不能从数据里解出来。答案是——放开一个从没人明说过的假设,它根本没有唯一解。

正文

一、这条论证卡在一个从没被估过的量上

前几篇的链条是:判读者的尺子会在两个方向歪——长度(尺度 a)和零点(位置 b)。尺度错有天花板,位置错没有;十二分之一标准差的零点偏移,就足以吃掉 5 个百分点那条判据线。所以「训练到底改没改到该改的地方」全押在 b 上。

问题是 b 从来没被估过。不是没人愿意,是这条论证停在一个假设上——「只要同一批受试者被 ≥2 位读者打分,b 就能估出来」。这话听着像 many-facet Rasch 里 severity 参数的现成搬用(Myford & Wolfe 那两篇经典综述确实把 severity 立为标准 facet,PMID 14523257 / 15064538)。但「有参数」和「参数可估」是两件事。本文用最朴素的潜变量阈值模型把它算一遍:真值 x~N(0,1),读者 j 把 a·x+b+ε 按自己的一组切点切成 K 级。

二、硬约束:切点一放开,b 没有唯一解

同一份合成数据(n=480,参照读者 b=0,第二位读者真 b=0.30,5 级),只改优化起点,别的都不动。

起点 b=0.00 收敛到 0.000,0.20 到 0.170,0.40 到 0.358,0.70 到 0.728,1.00 到 1.007。五个答案,负对数似然全部是 1460.4372,一位小数都不差。

不是优化器不收敛,是目标函数在这个方向上本来就平。把 b 固定在 0.00 至 0.90 六个数上、每次让切点组重估到最优,profile 似然一字不差。原因很直白:b 整体加 Δ,四个切点也整体加 Δ,每个受试者落进哪一级完全不变。b 与切点组是一条共线的平移方向,这条方向上没有信息。

这不是精度问题,是「有没有答案」的问题。它直接作废一种很自然的实验设计:训练前后各判一轮、每轮切点各自自由,然后比两轮 b。这么跑出来的数,取决于优化器从哪出发。

三、要救,得加一个与目的相冲突的假设

唯一救法是假定两轮共享同一组切点——「训练没改分档边界,只改了整体松紧」。加上这条,profile 立刻有唯一极小:真值 0.30 处最低,0.15 处差 2.86,0.45 处差 1.65,两头都掉出 95% 区间。

但请看清它在说什么:它假定训练没动切点。而「训练把判读标准教对了」恰恰是判读训练最主要的机制之一,也是本系列要检测的对象。用一个「训练没改边界」的假设,去检验「训练改了什么」,逻辑上是循环的。做了之后结论只能写成条件句,而这个前提得先独立验证。

另一条路本文没走但值得记:把时间维显式建模。Uto 的贝叶斯 MFRM 用马尔可夫先验绑住相邻时点的 severity,实测比各时点独立估计更准(PMID 36284065)。它不消除共线,但用「相邻时点不该跳变」的先验把方差压住。前提是得有时序,不是两轮。

四、就算可识别,精度也差一倍

共享切点这个可识别版本上,把样本量拉一遍(每档 120~150 次重复):n=120 时 b̂ 标准差 0.140、95% 置信区间半宽 0.275;n=240 是 0.096 / 0.188;n=480 是 0.070 / 0.145;n=960 是 0.051 / 0.091。收敛速度约 1.55/√n。

对照上一轮立的判据:「n=120 时 b̂ 的 95% 半宽超过 0.13 就说明不可测」。实测 0.275,是 0.13 的 2.1 倍——判据触发,而且不是刚超线。换句话说,那条判据自己把 n=120 判了死刑,它同时又把这档写成实验起点,两句话不能同时成立。

要把半宽压到 0.126,需 n≈580;只要求标准差达标,需 n≈151。「120 例起步」在任何一档上都不够。

更麻烦的一层:这些数还是把切点当已知算的。真实分析里切点也得估,b 的方差再涨一截——n=480 时从 0.060 到 0.070(1.17 倍),n=2000 时从 0.030 到 0.039(1.27 倍)。折成等效样本量,n=480 只顶 655 例,掉近两成。

五、那个「自由度来自读者数」的推论,实测是反的

上一轮留了一句处方:「要精度就加读者,不加病例,因为 b 的自由度来自读者数。」这句话在两个意义上都错。

第一层,读者确实提供配对自由度——没有第二位读者,b 与切点连共线都谈不上。所以「必须 ≥2 位读者」是对的。但配对只解决有无解,不解决解得准不准。

第二层才是要害。固定总判读次数 480,让 R 位读者各评同一批病例:R=2、每读者 240 例 → 0.055;R=4、120 例 → 0.085;R=8、60 例 → 0.134;R=16、30 例 → 0.210。读者从 2 位加到 16 位,精度掉了 3.8 倍。每增一位读者都在摊薄每人的病例数,而精度只看「这一位读者评了多少例」。

「每人重复看 2 次」也救不了:总判读 480 次下,240 例×2 遍得 0.086,120 例×4 遍得 0.120,60 例×8 遍得 0.143。优先级是病例数 > 重复次数 > 读者数,与上一轮正好倒过来。

六、b 到 δ 的斜率不是常数,是切点处的密度

上一轮换算用中位切点那档,得「b=0.3 → 10.6pp、0.6 → 20.4pp、1.5 → 41.0pp」,据此说「十二分之一标准差就吃掉整条线」。这个系数不是常数,它等于切点处的标准正态密度 φ(c),而 c 由阳性率决定。

按「δ=5pp 需要多大 b」反解:阳性率 50% 要 0.126,40% 要 0.128,30% 要 0.139,20% 要 0.167,10% 要 0.245,5% 要 0.363,2% 要 0.578。低阳性率任务里,「撞线」需要的位置偏移是中位场景的 2.9 到 4.6 倍。

这对「气功外气变化」这类判读特别要紧:它多半不是对半分的人工二分类,阳性事件本来就稀少。在高门槛场景,那句「小到没人当回事」的 0.126σ 反而真不算大事——要 0.36 到 0.58 个标准差才够。同一个数,中位切点下是警报,2% 阳性率下是常态噪声。

顺带一个好消息:上一轮担心「a 与 b 高度相关(abs(r)>0.7)就不可分」,实测远没到。n=120 时 r=0.035,n=480 时 0.202,n=1920 时 0.161。两个自由度数值上是分开的,那条担心不成立。

证据与局限

已验证:b 与切点组的平移共线是模型的结构性质,profile 似然在 b 方向严格平坦(六点扫描 ΔNLL 全为 0.00000),多起点收敛到 0.000/0.170/0.358/0.728/1.007 而似然同为 1460.4372;共享切点后唯一可识别(0.15 与 0.45 处 ΔNLL 各 2.86 / 1.65,均超 95% 界);b̂ 精度按约 1.55/√n 收缩,n=120 半宽 0.275;固定总判读次数下读者数从 2 增到 16,b̂ 散布从 0.055 涨到 0.210;切点自由估计使 b 方差膨胀 1.17~1.27 倍;b→δ 换算斜率等于 φ(c),低阳性率下所需 b 放大 2.9~4.6 倍。

部分验证:全部数字来自潜变量阈值模型的合成数据(5 级、等距真值、正态噪声),未在真实判读数据上复现。模型假设「读者只经 a、b 两个自由度作用于真值」,真实判读者还有中心化倾向、晕轮效应等额外 facet(PMID 14523257 列了五类),会额外增加方差,故本文精度数字应视为上限(乐观侧)。MFRM 的 severity 与本文的 b 概念对应,但前者用 logit 尺度、后者用真值标准差尺度,数值不可直接互换。

待验证:「切点未被训练改变」这个可识别性前提在真实判读数据上是否成立,无任何实测;若训练确实动了切点,本文全部 b̂ 都是条件估计。真实判读任务的阳性率分布(决定 φ(c)、进而决定「5pp 需多大 b」)在本领域从未报告,本文只能给跨基率换算表。气功外气判读的等级数 K 也未定,K 由 3 增到 6 时 b̂ 散布从 0.108 降到 0.092(约 15% 改善),但 K 越多各等级越薄,可行性未知。

L1 网络状态:L1 中对应本主题的是边表 E-060「标签前提」,状态为「冲突(未裁决),且新增两处单位错配/精度区间错配」——原文记「中医医师间诊断一致性在开放人群中『低』(PMID 30383398);同系列指出偶然校正一致性统计常被误用、建议 Gwet AC2(PMID 31730403 / 31634000 / 31730404);但训练可改善(PMID 19538100)」。冲突双方即:一致性低 vs 训练可改善。本文结论要求两边都成立才讲得通——「一致性低」对应 b 与切点共线导致分不开,而「训练可改善」若成立则恰好破坏本文的可识别性前提(训练动了切点)。故本文不判定该冲突,只指出冲突里有一层从没被分开:两个观察都可以由「一致率 P_o 变化」单独造出,而不动切点分毫。(就「位置参数的可估性、共线方向、精度归属」这一层而言,该主题尚未进入 L1 对应网络,已在 science-yixue-l1-gaps.md 登记为「测量一致性」簇第 34 条。)

相关阅读

证据等级: C
参考文献: PMID 14523257, PMID 15064538, PMID 36284065, PMID 33169286, PMID 20473138, PMID 8501467, PMID 30383398, PMID 19538100

证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Estimability of Position Drift Collapses Before Its Accuracy

Evidence level: C

The previous article calculated “how much position drift causes crossing the decision line.” This article asks: can that drift magnitude itself be solved from the data? The answer is—by relaxing an assumption never explicitly stated before, there is no unique solution.

Body

I. This Argument Stalls on a Quantity Never Previously Estimated

The chain in previous articles is: the rater’s scale tilts in two directions—length (scale parameter $a$) and zero-point (position parameter $b$). Scale error has a ceiling; position error does not. A zero-point shift of just one-twelfth of a standard difference consumes 5 percent of that decision line. Thus, whether “training actually changed what needed changing” hinges entirely on $b$.

The problem is that $b$ has never been estimated. Not because no one was willing, but because this argument stops at an assumption—that “as long as the same subjects are rated by ≥2 raters, $b$ can be estimated.” This sounds like a ready application of the severity parameter in many-facet Rasch models (Myford & Wolfe’s classic reviews indeed establish severity as the standard facet, PMID 14523257 / 15064538). But “having a parameter” and “being estimable” are two different matters. This paper computes it using the simplest latent variable threshold model: true values $x \sim N(0,1)$, and rater $j$ cuts $a \cdot x + b + \varepsilon$ into $K$ levels according to their own set of cutpoints.

II. Hard Constraint: Once Cutpoints Are Freed, $b$ Has No Unique Solution

Using the same synthetic data ($n=480$, reference rater $b=0$, second rater true $b=0.30$, 5 levels), we only change the optimization starting point, leaving everything else fixed.

Starting at $b=0.00$ converges to 0.000, 0.20 to 0.170, 0.40 to 0.358, 0.70 to 0.728, 1.00 to 1.007. Five answers, yet all negative log-likelihoods are exactly 1460.4372, differing not even in the first decimal place.

It is not that the optimizer fails to converge; the objective function is inherently flat in this direction. Fixing $b$ at six values from 0.00 to 0.90 and re-optimizing the cutpoints each time yields profile likelihoods identical to a single decimal. The reason is straightforward: adding $\Delta$ to $b$ overall also adds $\Delta$ to all four cutpoints, so each subject falls into exactly the same level. $b$ and the cutpoint group lie on the same collinear translation direction; there is no information along this direction.

This is not a precision issue, but a “whether an answer exists” problem. It directly invalidates a very natural experimental design: run judgments before and after training, with each round’s cutpoints freely optimized, then compare the two $b$ values. The numbers produced depend entirely on where the optimizer starts.

III. To Save It, We Must Add an Assumption That Conflicts with the Goal

The only rescue is to assume both rounds share the same cutpoints—”training did not change the grading boundaries, only the overall tightness.” Adding this constraint immediately yields a unique minimum: lowest at true value 0.30, off by 2.86 at 0.15, off by 1.65 at 0.45, with both tails falling outside the 95% interval.

But please note what this implies: it assumes training did not move the cutpoints. Yet “training teaching the judgment standard correctly” is precisely one of the main mechanisms of judgment training, and also the object this series aims to detect. Using an assumption that “training did not change boundaries” to test “what training changed” is logically circular. Afterward, conclusions can only be written as conditional statements, and this premise must first be independently verified.

Another path not taken here but worth noting: explicitly modeling the time dimension. Uto’s Bayesian MFRM uses Markov priors to bind severity across adjacent time points, empirically outperforming independent estimates at each point (PMID 36284065). It does not eliminate collinearity, but uses the prior that “adjacent time points should not jump” to suppress variance. The prerequisite is having a time series, not just two rounds.

IV. Even If Identifiable, Precision Is Twice as Poor

On this identifiable version with shared cutpoints, we pull the sample size through (120~150 repetitions per level): $n=120$ gives $\hat{b}$ standard deviation 0.140, 95% CI half-width 0.275; $n=240$ is 0.096 / 0.188; $n=480$ is 0.070 / 0.145; $n=960$ is 0.051 / 0.091. Convergence rate is approximately $1.55/\sqrt{n}$.

Comparing to the decision criterion set earlier: “when $n=120$, if $\hat{b}$’s 95% half-width exceeds 0.13, it is unmeasurable.” The actual 0.275 is 2.1 times 0.13—the criterion triggers, and not barely. In other words, that criterion itself sentenced $n=120$ to death, while simultaneously using it as the experimental starting point; these two statements cannot both hold.

To press the half-width down to 0.126, we need $n \approx 580$; requiring only standard deviation to meet the mark, we need $n \approx 151$. “Starting with 120 cases is insufficient for any level.”

A more troubling layer: these numbers treat cutpoints as known. In real analysis, cutpoints must also be estimated, inflating $b$’s variance further—by 1.17 times when $n=480$ (from 0.060 to 0.070), and by 1.27 times when $n=2000$ (from 0.030 to 0.039). Converted to equivalent sample size, $n=480$ only equals 655 cases, dropping nearly 20%.

V. The Inference That “Degrees of Freedom Come from Number of Raters” Is Actually Reversed

The previous article left a prescription: “for precision, add more raters, not more cases, because $b$’s degrees of freedom come from the number of raters.” This is wrong in two senses.

First, raters do provide paired degrees of freedom—without a second rater, $b$ and cutpoints cannot even be collinear. So “must have ≥2 raters” is correct. But pairing only solves whether a solution exists, not how accurate it is.

The second point is the crux. Fixing total judgment count at 480, let $R$ raters each rate the same batch of cases: $R=2$, 240 cases each → 0.055; $R=4$, 120 cases each → 0.085; $R=8$, 60 cases each → 0.134; $R=16$, 30 cases each → 0.210. Increasing raters from 2 to 16 reduces precision by 3.8 times. Each additional rater dilutes the number of cases per person, while precision depends only on “how many cases this rater judged.”

“Each person repeats twice” also fails: under total 480 judgments, 240 cases × 2 rounds gives 0.086, 120 cases × 4 rounds gives 0.120, 60 cases × 8 rounds gives 0.143. Priority is cases > repetitions > raters, exactly the reverse of the previous claim.

VI. The Slope from $b$ to $\delta$ Is Not Constant; It Equals Density at the Cutpoint

The previous conversion used the median cutpoint level, yielding ” $b=0.3 \to 10.6\text{pp}$, $0.6 \to 20.4\text{pp}$, $1.5 \to 41.0\text{pp}$,” hence claiming “one-twelfth of a standard difference eats the whole line.” This coefficient is not constant; it equals the standard normal density $\phi(c)$ at the cutpoint, where $c$ is determined by the positive rate.

Solving backwards for “what $b$ is needed for $\delta=5\text{pp}$”: positive rate 50% requires 0.126, 40% requires 0.128, 30% requires 0.139, 20% requires 0.167, 10% requires 0.245, 5% requires 0.363, 2% requires 0.578. In low-positive-rate tasks, the position shift needed to “hit the line” is 2.9 to 4.6 times the median scenario.

This is especially critical for judgments like “qigong external qi changes”: they are rarely artificial binary splits with 50% positivity, since positive events are inherently scarce. In high-threshold scenarios, that “tiny 0.126σ nobody cares about” is actually not a big deal—it requires 0.36 to 0.58 standard differences to matter. The same number is an alarm under median cutpoints but normal noise under 2% positive rate.

A side note of good news: the previous worry that “if $|r(a,b)| > 0.7$, they are inseparable” never materialized in practice. At $n=120$, $r=0.035$; at $n=480$, $r=0.202$; at $n=1920$, $r=0.161$. The two degrees of freedom are numerically separable, so that worry does not hold.

Evidence and Limitations

Verified: $b$’s collinearity with the cutpoint group is a structural property of the model; profile likelihood is strictly flat in the $b$ direction (six-point scan $\Delta\text{NLL}$ all 0.00000); multiple starting points converge to 0.000/0.170/0.358/0.728/1.007 with identical likelihood 1460.4372; after sharing cutpoints, only identifiable (ΔNLL at 0.15 and 0.45 are 2.86 / 1.65, both exceeding 95% bounds); $\hat{b}$ precision contracts at rate ~1.55/√n, half-width 0.275 at $n=120$; under fixed total judgments, increasing raters from 2 to 16 inflates $\hat{b}$ spread from 0.055 to 0.210; free estimation of cutpoints inflates $b$ variance by 1.17~1.27 times; the $b \to \delta$ conversion slope equals $\phi(c)$, requiring $b$ to be 2.9~4.6 times larger under low positive rates.

Partially Verified: All numbers come from synthetic data of the latent variable threshold model (5 levels, equally spaced true values, normal noise), not replicated on real judgment data. The model assumes “raters affect true values only via $a$ and $b$ degrees of freedom,” whereas real raters have additional facets like central tendency bias and halo effect (PMID 14523257 lists five types), which would further increase variance; thus these precision figures should be viewed as upper bounds (optimistic side). MFRM’s severity corresponds to this paper’s $b$, but the former uses logit scale while the latter uses true-value standard deviation scale, so values cannot be directly swapped.

To Be Verified: Whether the identifiability premise “cutpoints remain unchanged by training” holds for real judgment data has no empirical verification; if training indeed moved cutpoints, all $\hat{b}$ here are conditional estimates. The positive rate distribution of real judgment tasks (which determines $\phi(c)$, hence “what $b$ is needed for 5pp”) has never been reported in this field; this paper can only provide cross-rate conversion tables. The number of levels $K$ for qigong external qi judgment is also undetermined; when $K$ increases from 3 to 6, $\hat{b}$ spread drops from 0.108 to 0.092 (~15% improvement), but with more levels each grade becomes thinner, feasibility unknown.

L1 Network Status: In L1, the entry corresponding to this topic is edge table E-060 “Label Premise”, status “Conflict (Unresolved), with two new unit mismatches/precision interval mismatches—original record states “diagnostic consistency among TCM physicians is ‘low’ in open populations (PMID 30383398); the same series notes that chance-corrected consistency statistics are often misused, recommending Gwet AC2 (PMID 31730403 / 31634000 / 31730404); yet training can improve (PMID 19538100).” The conflicting parties are: low consistency vs. trainable improvement. This paper’s conclusion requires both to hold for coherence—”low consistency” corresponds to $b$ and cutpoints being collinear and inseparable, while “trainable improvement” if true would exactly destroy this paper’s identifiability premise (training moved cutpoints). Hence this paper does not resolve the conflict, only points out one layer within it never separated: both observations can be generated solely by “consistency rate $P_o$ change” without moving cutpoints at all.** (Regarding this layer of “position parameter estimability, collinear direction, precision attribution,” this topic has not yet entered the corresponding L1 network; it is registered in science-yixue-l1-gaps.md as cluster #34 of “measurement consistency”.)

Related Reading

Evidence Level: C
References: PMID 14523257, PMID 15064538, PMID 36284065, PMID 33169286, PMID 20473138, PMID 8501467, PMID 30383398, PMID 19538100

Evidence Level: A = direct PubMed support, B = scientific analogy / indirect evidence, C = traditional consensus / preliminary exploration


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *