Evidence level: C
锚校正该不该用,取决于资深者与普通读者的误差是否同向(ρ 的符号)。这个符号不必做扰动实验:两份现成的五档边际分布一比对,门限往哪边挪了,符号就定了。
正文
一、设问的最后一环:符号从哪来
上一篇(#1460)把锚校正的账算清了:锚误差 h 与读者误差 e 的相关系数 ρ 为正时,加锚有收益窗口(τ ∈ [ρ(√m−1), ρ(√m+1)],谷底 1−ρ²);ρ 为负时锚是灾难(ρ=−0.5、τ=1.414 时相对方差 5.0,是独立情形的 1.7 倍)。但真实研究里 ρ 从未被测过,而它由「两类评分者的偏倚是否同向」决定:资深者判「变化够大」的门限更低(看得更细),普通读者门限更高,同一份材料上一个偏正一个偏负,ρ<0;两者共用同一套不成文门限、只是噪声大小不同,则 ρ>0。问题在于:门限位置看不见,怎么比?
二、门限是窗,边际比例是它的影子
五档量表的评分机制可以写成:评分者内心有个连续量,过一道道门槛,落到第几档就报第几档。门槛本身观测不到,但它留了影子——每档的边际占比。关键在于:同一批材料由两组人各判一遍时,底层的真实量对两组是同一个(判的是同一批),于是两组边际分布的任何系统性差异,只能来自门槛位置或噪声尺度。这正是多面 Rasch 测量反复证实的现象:评分者「严厉度」差异普遍存在且幅度可观——54 个评审组 × 3000 名考生的大学入学评分量出大而不一致(PMID 40558845);五点量表上直接观察到评分者用分位置差异、须把评分者效应建进模型(PMID 42841527)。影子既然存在,就能反推窗子挪向哪边。
三、方向判据:一阶随机占优
判据是:把两组的累计分布画出来,若资深者的 CDF 在所有切点上都不高于普通读者(高档占比系统性更高),则资深者门限整体更低——一阶随机占优。本轮用 20 万次模拟核验:门槛基准 [−1.5, −0.5, 0.5, 1.5],资深者整体下移 1.4 档时,其五档边际为 [0.018, 0.117, 0.324, 0.358, 0.184],普通读者为 [0.240, 0.377, 0.285, 0.087, 0.011],四个切点的 CDF 差全部为负(−0.22 到 −0.48)——占优方向与门限平移方向严格一致;下移 0.5 档的温和情形占优仍成立(四切点 −0.04 到 −0.19)。反过来,资深者高档占比系统性更高,就等价于「同一份边缘病例,资深者更愿意报重」——这正是两类误差反号的结构条件,ρ<0。
四、关键对照:占优失败不等于没有差异
纯噪声差异(门槛相同、只是资深者噪声更小)下两组 CDF 会交叉:模拟中噪声比 1.6 比 1 时,CDF 差在低段 +0.107、高段 −0.107,符号翻转,不构成占优。所以「无占优」排除的是门限平移的主导地位,不是一切差异;此时 ρ>0 路线成立(误差同向),锚校正可用。这也解释了为什么不能拿「均值差」当门限差用:均值差不区分「平移」与「散布变化」,占优检验区分。信号检测文献里把判据位置与辨别力分开估计,正是同一逻辑(PMID 23784010);医学判读上新手与算法/专家的一致性差异也反复被量化(PMID 42278802 新手目测与无标记动捕分类的对比;PMID 42474519 评阅者间膝关节数字分级一致性)。而评分者效应研究同时警告:训练不必然消除严厉度差(PMID 40471819:训练只在部分维度上缩小了评分者差异)——所以先读分布再决定校不校,比先训练后祈祷更省。
五、落到锚校正的三分叉
①资深者分布一阶占优且门限平移量大于 0.3 档宽 ⇒ ρ<0,锚校正在本场景不适用,改「先按经验分层、各层自校准」;②分布交叉、仅散布不同 ⇒ ρ>0,加锚成立,且按 #1460 的收益窗口可放宽 τ;③占优但平移小于 0.1 档宽 ⇒ 视为门限共用,按噪声差异处理,ρ 的上界由 1/√R 给出。整套判据的全部原料——两组各自的五档占比——在任何已完成的判读研究里都是现成的,一分钱新实验都不用花。
证据与局限
已验证: 门限-边际映射的数学结构(序数门限模型恒成立);一阶随机占优对门限平移方向的充分性(本轮 20 万次模拟,平移 0.5 与 1.4 档两情形四切点全部同号)。
部分验证: 评分者严厉度差异的普遍性(多面 Rasch 多项实测:PMID 40558845、PMID 42841527、PMID 40471819);占优方向在小平移下受抽样噪声影响,需 Mann-Whitney 类检验与足够样本量(建议每组 ≥15 人、≥120 例)支撑。
待验证: ①「占优方向 ⇒ ρ<0」的映射在同一批材料双组判读数据上的实测(本判据的核心外推);②门限平移量(档宽单位)与 ρ 的绝对值的定量关系;③两组同时存在平移与散布差时,占优方向是否仍由平移主导(模拟显示平移主导时仍占优,但边界条件未定);④资深者门限更低是否等于「误差反号」——若资深者的高档占比差异来自真实的辨别力而非门限,符号判断须重推。
相关阅读
L1 网络状态:该主题尚未进入 L1 对应网络。CORRESPONDENCE_NETWORK.md(2037 行)内检索「门限/阈值/偏倚/评分者/序数/占优」仅命中 E-060 相邻边(第 202 行 E-060「标签前提」conflict:中医医师间诊断一致性低 PMID 30383398 vs 训练可改善 PMID 19538100,未裁决;第 260 行 KB-333 conflict:κ 型系数对序数分类的稳健性方向与 Gwet 一系相反;第 261 行 KB-334:阳性率不同时一致性系数与患病率须分报)——均为「标签信度」簇的相邻边,与「评分者门限位置差 × 误差相关符号 ρ」非同一边;边表(63 行)无对应边。按硬规则证据等级封顶 C,缺口已登记。
证据等级: C
参考文献: PMID 40558845, PMID 42841527, PMID 40471819, PMID 23784010, PMID 42278802, PMID 42474519
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
The sign of the anchor error is directly readable from existing data: whichever of the two raters’ five-level distributions exerts more pressure determines the direction of the…
Evidence level: C
Whether the correlation coefficient ρ between the anchor error $h$ and reader error $e$ is positive or negative depends entirely on which group of raters exerts more “pressure” on the rating scale. The sign of ρ can be read directly from existing data by comparing the five-tier marginal distributions of the two groups.
Whether to apply anchor correction depends on whether the errors of senior raters and ordinary readers are aligned (the sign of ρ). This sign does not require perturbation experiments: simply comparing two existing five-tier marginal distributions reveals which side the threshold shifts toward, thereby determining the sign.
Body
I. The Final Step of Questioning: Where Does the Sign Come From?
The previous entry (#1460) settled the accounting for anchor correction: when ρ is positive, adding an anchor yields a benefit window (τ ∈ [ρ(√m−1), ρ(√m+1)], with a valley of 1−ρ²); when ρ is negative, the anchor becomes a disaster (e.g., at ρ=−0.5 and τ=1.414, the relative variance is 5.0, which is 1.7 times that of the independent case). However, in real-world studies, ρ has never been directly measured. Instead, it is determined by whether the biases of the two types of raters point in the same direction: senior raters set a lower threshold for judging “sufficient change” (seeing more detail), while ordinary readers set a higher threshold; if the same material is rated such that one group leans positive and the other negative, then ρ<0; if both groups share the same unwritten threshold but differ only in noise magnitude, then ρ>0. The problem is: since the threshold position is invisible, how can we compare them?
II. Thresholds Are Windows; Marginal Proportions Are Their Shadows
The rating mechanism of a five-tier scale can be written as: each rater has an internal continuous quantity that passes through successive thresholds, landing in the corresponding tier. The thresholds themselves cannot be observed directly, but they leave behind shadows—the marginal proportions in each tier. Crucially, when the same batch of materials is rated by two groups, the underlying true quantity is identical for both (since they are rating the same items); thus, any systematic difference between the two groups’ marginal distributions must arise solely from differences in threshold positions or noise scales. This is precisely the phenomenon repeatedly confirmed by multi-faceted Rasch measurement: differences in rater “severity” are ubiquitous and substantial—54 review groups × 3,000 examinees yielded large yet inconsistent university entrance scores (PMID 40558845); on five-point scales, direct observation shows raters use different cut-off positions, necessitating that rater effects be built into the model (PMID 42841527). Since these shadows exist, we can reverse-engineer which way the window has shifted.
III. Directional Criterion: First-Order Stochastic Dominance
The criterion is as follows: plot the cumulative distribution functions (CDFs) of both groups. If the senior rater’s CDF is never higher than that of the ordinary reader at any cut-off point (i.e., the proportion in higher tiers is systematically greater), then the senior rater’s threshold is overall lower—this constitutes first-order stochastic dominance. In this round, 200,000 simulations verified this: with baseline thresholds [−1.5, −0.5, 0.5, 1.5], when the senior group’s thresholds shift downward by 1.4 tiers, their five-tier marginal distribution becomes [0.018, 0.117, 0.324, 0.358, 0.184], while the ordinary group’s is [0.240, 0.377, 0.285, 0.087, 0.011]; the CDF differences at all four cut-off points are negative (−0.22 to −0.48)—the direction of dominance strictly matches the direction of threshold shift; even in the gentler case of a 0.5-tier downward shift, dominance still holds (CDF differences at the four cut-off points range from −0.04 to −0.19). Conversely, if the senior group’s proportion in higher tiers is systematically greater, this is equivalent to “for the same borderline case, the senior rater is more willing to rate it heavily”—this is precisely the structural condition where the two types of errors have opposite signs, implying ρ<0.
IV. Key Contrast: Failure of Dominance Does Not Mean No Difference
Under pure noise differences (same thresholds, but the senior group has smaller noise), the two groups’ CDFs will cross: in simulations, when the noise ratio is 1.6 to 1, the CDF difference is +0.107 at the low end and −0.107 at the high end, causing the sign to flip, thus failing dominance. Therefore, “no dominance” excludes only the dominance of threshold shifting, not all differences; in such cases, the ρ>0 route holds (errors are aligned), and anchor correction is applicable. This also explains why one cannot use “mean difference” as a proxy for “threshold difference”: mean differences do not distinguish between “shift” and “spread change,” whereas dominance tests do. Signal detection literature separates the estimation of criterion location from discriminability, following the same logic (PMID 23784010); medical diagnosis repeatedly quantifies consistency differences between novices and algorithms/experts (PMID 42278802 novice visual estimation vs. markerless motion capture classification; PMID 42474519 inter-rater consistency in knee digital grading). Meanwhile, rater effect research simultaneously warns that training does not necessarily eliminate severity differences (PMID 40471819: training only reduces rater differences on some dimensions)—thus, reading the distribution first to decide whether to calibrate is more efficient than training first and then praying.
V. The Three-Way Fork of Anchor Correction
① If the senior group’s distribution exhibits first-order dominance and the threshold shift exceeds 0.3 tier widths ⇒ ρ<0, so anchor correction is inappropriate in this scenario; instead, adopt “stratify by experience first, then self-calibrate each layer”; ② If the distributions cross with only spread differences ⇒ ρ>0, so adding an anchor is valid, and per the benefit window in #1460, τ can be relaxed; ③ If dominance exists but the shift is less than 0.1 tier widths ⇒ treat as shared thresholds, handle via noise differences, with the upper bound of ρ given by 1/√R. All raw materials for this entire criterion—the five-tier proportions of each group—are readily available in any completed rating study, requiring not a single penny of new experimentation.
Evidence and Limitations
Verified: The mathematical structure of threshold-marginal mapping (ordinal threshold models always hold); the sufficiency of first-order stochastic dominance for determining the direction of threshold shift (in this round, 200,000 simulations confirmed that for shifts of 0.5 and 1.4 tiers, the CDF differences at all four cut-off points retained the same sign).
Partially Verified: The universality of rater severity differences (multi-faceted Rasch measurements confirmed this: PMID 40558845, PMID 42841527, PMID 40471819); the direction of dominance under small shifts is susceptible to sampling noise, requiring Mann-Whitney-type tests and sufficient sample sizes (recommended ≥15 per group, ≥120 cases) to support robust inference.
Pending Verification: ① Empirical validation of the mapping “dominance direction ⇒ ρ<0” on dual-group rating data from the same batch of materials (this is the core extrapolation of this criterion); ② The quantitative relationship between threshold shift magnitude (in tier-width units) and the absolute value of ρ; ③ Whether dominance direction remains governed by shift when both groups exhibit simultaneous shift and spread differences (simulations show shift dominates when present, but boundary conditions remain undetermined); ④ Whether lower senior thresholds equate to “opposite-signed errors”—if the senior group’s higher-tier proportion differences stem from genuine discriminability rather than threshold effects, the sign judgment must be re-derived.
Further Reading
- When Anchor and Reader Share One Pair of Eyes, Error Is Not Amplified but Cancelled
- Does “Using Seniors as Reference” Go Wrong? The Question Is Not About Frequency
- Adding Scale Marks to the Ruler Is Right, But the Calibrators Only Sample One-Tenth
- Resonance Consciousness Field: Qi Gong External Qi Research Pool
L1 Network Status: This topic has not yet entered the L1 corresponding network. Searching for “threshold/limit/bias/rater/ordinal/dominance” in CORRESPONDENCE_NETWORK.md (2,037 lines) only hits E-060 adjacent edges (line 202, E-060 “label premise” conflict: low diagnostic consistency among TCM physicians PMID 30383398 vs. training improving it PMID 19538100, unresolved; line 260, KB-333 conflict: κ-type coefficient robustness direction for ordinal classification opposes Gwet’s series; line 261, KB-334: when positivity rates differ, consistency coefficients and prevalence must be reported separately)—all belonging to the “label reliability” cluster, not the same edge as “rater threshold position difference × error correlation sign ρ”; the edge table (63 lines) lacks the corresponding edge. Hard-rule evidence level capped at C; gap registered.
Evidence Level: C
References: PMID 40558845, PMID 42841527, PMID 40471819, PMID 23784010, PMID 42278802, PMID 42474519
Evidence Level: A = direct PubMed support, B = scientific analogy/indirect evidence, C = traditional consensus/preliminary exploration
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply