Evidence level: C
一个量测不出来,不代表它的后果算不出来。上一轮想把 ε 拆成「稳定的个人风格」与「同一人的试次摇摆」,并打算用「谁占六成」来决定死区判据要不要用。三刀下去,这个分解不但做不出来,而且拆开之后的方向跟原计划完全相反。
正文
上一篇(kappa-marginal-tvd-deadband-closed-form)把整条一致率天花板的推导收成一个式子:ε* = δ/(4(1−m̄))。m̄ 是主档占比,δ 是两位读者主档占比的差。ε 一旦越过 ε*,总变差距离 TVD 就开始超过 δ,天花板被压得更低。但 ε 从头到尾没人实测过——上一篇只能写「若 ε≤0.05 则安全」,说不出真实研究里这个数长什么样。
于是有了一个看起来非常自然的计划:找一批 ≥120 例、≥2 位读者、每人重复判读 ≥2 次的三档数据,把 ε 的方差拆成两层——「人×稳定」(这人一贯偏乐观)和「人×病例×试次」(同一批片子看两遍自己就不一致)。按原设想:若稳定成分 >60%,死区判据可用;若随机成分 >60%,死区警告可降级为误报。
第一刀:这个分解的自由度只有读者数减一。
稳定成分的方差 σ²_a 是靠「各读者的长期均值之间有多大差异」估出来的。k 位读者,就只有 k 个均值,df = k−1。这个自由度跟你收多少病例、每人判多少次,完全无关。k=2 时 df=1,σ²_a 的相对标准差是 141%,w_s(稳定成分占比)的 90% 置信区间半宽 ±52.9 个百分点——这个区间宽到能把任何一个真值同时判成「稳定主导」和「随机主导」。要到 ±10pp 需要 k≥30 位读者,±5pp 需要 k≥114 位。
气功外气这类研究,判读者通常就是两三位老师。在这个配置下,w_s 是一个原理上估不准的数,不是「样本量不够再补点」。
文献侧也印证了这一点。Kreiter 与 Wilson(PMID 26925540)明确指出:在嵌套设计下,系统性的评分者效应与「评分者×被评者」交互效应会被合并成一个混杂的方差成分,无法分开估计;要分开只能靠完全交叉模型。这不是统计技巧问题,是数据结构本身不给这个信息。而 Svensson 与 Holm(PMID 7701145)早在 1994 年就给出了一套把有序分类配对变异分为系统(由边际决定)与随机(相对秩方差)的方法——方法存在,但对两位读者的情形,系统部分的估计精度同样被 df 卡死。Sturman 等(PMID 28587597)追踪考官严厉度发现,即便没有任何反馈干预,严厉度在几个月内也会明显漂移;他们自己都说「这种固有的不稳定性使考官训练策略的研究变得复杂」。也就是说,「稳定的个人风格」这个前提本身就未必成立——它可能只是一个移动得足够慢、被误当成常数的变量。
第二刀:就算测出来了,判决方向是反的。
这是本轮最要紧的发现。原计划的推理是:随机成分期望为零、不系统性地把 TVD 推高,所以随机占主导时死区可以不管。
但这个推理漏了一个符号问题:TVD 对 ε 是偶函数。三档下 TVD = max{|p₁−q₁|, |p₂−q₂|, |p₃−p₃|},而我们的参数化让次档差恰好正比于 |ε_A − ε_B|。也就是说,TVD 只吃两位读者偏离的差值绝对值,完全不吃这个差值的正负号。一位偏乐观、一位偏保守(符号相反,差值最大)会推高 TVD;两位都偏乐观、但程度不同(符号相同,差值非零)同样推高 TVD。所以「期望为零」在这里一点保护作用都没有。
量化一下:同方差 s 下,稳定成分(两点分布 ±s)的 E|Δ| = s;随机成分(高斯)的 Δ 服从 N(0, 2s²),E|Δ| = √(4/π)·s ≈ 1.128s。随机成分的抬升能力反倒高出 12.84%。
更要紧的是硬上界:如果两位读者的稳定风格都落在 |ε| ≤ ε* 区间内,那么 |Δ| ≤ 2ε*,TVD 恒等于 δ,永不越线。在 m̄=0.75/0.85/0.90 三档上把 |ε|≤ε* 的全网格扫了一遍,max(TVD) 精确等于 δ,误差在 10⁻¹² 量级。而随机成分服从连续分布,尾部永远能越过任意阈值:m̄=0.85、s=ε*=8.33% 时,纯稳定 |ε|=ε* 的越线率是 0.00%,纯随机 sd=ε* 的越线率是 16.50%(5 万次模拟)。
结论反过来了:稳定成分是安全的,随机成分才是危险的。原计划「随机占主导 ⇒ 死区警告降级为误报」这一条,方向错了。
第三刀:w_s 是尺度无关的,而天花板负担不是。
这可能是最根本的漏洞。w_s = σ²_a/(σ²_a+σ²_r) 是个比值,两组数据可以 w_s 完全相同而总方差差一个量级。实测(m̄=0.85,每组 2 万次):
| σ²_a | σ²_r | w_s | E[TVD] − δ |
|---|---|---|---|
| 0.65 | 0.35 | 0.65 | +11.44pp |
| 0.13 | 0.07 | 0.65 | +4.02pp |
| 0.0325 | 0.0175 | 0.65 | +1.21pp |
w_s 一模一样,天花板负担差了 9.5 倍。用 w_s 做判决,等于用一个无量纲比值去管一个有量纲的后果——量纲就不对。
还有一个自相矛盾:原计划说「重复判读越多,随机成分被平均掉,稳定成分占比越高,死区判据越该收紧」。但重复判读恰恰是降负担的手段。r 从 1 增到 10,w_s 从 35.0% 升到 84.3%(按 σ²_a=0.35、σ²_r=0.65 算),可同一过程中纯随机的越线率从 16.34% 掉到 0.00%。也就是说,「w_s 升高」这件事本身正是「风险下降」的副产品,把它读成「该更警惕」,逻辑反了。
替身:不拆,直接测那个差值。
既然 TVD 只吃 |Δ|=|ε_A−ε_B|,那要判的本来就是一个数。判据直接写:|Δ| ≤ 2ε* = δ/(2(1−m̄))。m̄=0.85 时这条线是 16.67%,m̄=0.75 时是 10.00%,m̄=0.90 时是 25.00%。
这是标准的非劣性检验,所需病例数(单侧 α=.05,次档基准比例 p=0.25):
| m̄ | 判据线 2ε* | n(r=1) | n(r=2) | n(r=3) |
|---|---|---|---|---|
| 0.70 | 8.33% | 147 | 74 | 49 |
| 0.75 | 10.00% | 102 | 51 | 34 |
| 0.80 | 12.50% | 65 | 33 | 22 |
| 0.85 | 16.67% | 37 | 19 | 13 |
| 0.90 | 25.00% | 17 | 9 | 6 |
对照一下:分解 w_s 的路子要 ≥40 位读者才有 80% 功效,而直接测 |Δ| 只需要两位读者、几十到一百多例。同一笔研究预算,一个做得出来,一个做不出来。
证据与局限
已验证:在三档对称参数族 m_A=m̄±δ/2 下,TVD = max(三条边际差绝对值),且次档差正比于 |ε_A−ε_B|——这是纯代数,闭式在 ε=ε* 处精确给出 TVD=δ(三档 m̄ 逐点核验,误差 <10⁻¹²)。|ε|≤ε* 全网格上 max(TVD)=δ,稳定成分的硬上界成立。同方差下随机成分 E|Δ| 比稳定成分高 12.84%(√(4/π)),5 万次模拟越线率 16.50% vs 0.00%。w_s 相同而负担差 9.5 倍,实测三组一致。所需样本量按非劣性公式反解,为解析值。
部分验证:σ²_a 的 df=k−1 与相应的置信区间半宽是标准方差成分理论(正态近似),未做小样本精确分布的校正;w_s 的区间宽度用 delta 法一阶近似,在 k 很小时偏差会更大。文献侧只印证了「系统成分与交互成分在嵌套数据下不可分」「考官严厉度本身会漂移」这两大类结论,没有任何文献做过本文这个具体的「稳定 vs 随机对 TVD 抬升能力」的比较。重复判读只压随机成分这一设定是理想化的——实际重复判读可能引入记忆效应,把第二次判读变成第一次的复制,此时 r 的实际收益低于 √r。
待验证:真实三档裁决数据里 |Δ| 的实测分布;稳定成分是否真的「稳定」(Sturman 等的结果暗示可能不是,需要 ≥4 个时间点的追踪才能区分慢漂移与常数);记忆效应使重复判读收益折损多少;m̄ 与 δ 本身带抽样误差时 2ε* 这条线会怎么移动(#1405 专线)。
证据等级: C
参考文献: PMID 26925540, PMID 7701145, PMID 28587597, PMID 31169540L1 网络状态:CORRESPONDENCE_NETWORK.md 边表止于 E-050(纳音↔频率,
L1 | open,现代证据栏原文「尚无任何可测来源」);网络对齐指标行原文「L2 以上边数 | 0」。三域节点表无「判读 / 评分者 / 方差成分 / 一致率」任何登记位(全表检索「判读 / 裁决 / 一致率 / 边际 / 偏倚 / 方差 / 噪声 / 测量」零命中),物理侧「相位 / 节律」语义为「两振荡锁相」而非边际分布比较。故本文证据等级封顶 C,且该主题尚未进入 L1 对应网络(已登记至 l1-gaps)。
相关阅读
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Is ε the person’s temper or his shaky hand: an impossible decomposition and a possible stand-in
Evidence level: C
Measuring something does not imply its consequences cannot be calculated. In the previous iteration, we attempted to decompose ε into “stable personal style” and “trial-to-trial fluctuation of the same person,” intending to use “who accounts for 60%” to decide whether to apply the deadband criterion. After three cuts, this decomposition not only failed but reversed the direction of the split compared to the original plan.
Main Text
The previous article (kappa-marginal-tvd-deadband-closed-form) condensed the derivation of the consistency ceiling into a single formula: ε = δ/(4(1−m̄)). Here, m̄ is the proportion of the main category, and δ is the difference in main-category proportions between two readers. Once ε exceeds ε, the Total Variation Distance (TVD) surpasses δ, and the ceiling is pushed lower. However, ε has never been empirically measured; the previous article could only state “if ε ≤ 0.05, it is safe,” without describing what this number looks like in real research.
Thus, a seemingly natural plan emerged: collect three-category data from ≥120 cases, ≥2 readers, with each case rated ≥2 times, and decompose the variance of ε into two layers—”person × stable” (this person is consistently optimistic) and “person × case × trial” (the same set of images yields inconsistent results when viewed twice). According to the original hypothesis: if the stable component accounts for >60%, the deadband criterion applies; if the random component accounts for >60%, the deadband warning can be downgraded to a false alarm.
Cut 1: The degrees of freedom for this decomposition equal the number of readers minus one.
The variance of the stable component, σ²_a, is estimated by “how large the differences are between the long-term means of each reader.” With k readers, there are only k means, so df = k−1. This degree of freedom is completely independent of how many cases or repetitions per person are collected. When k=2, df=1, and the relative standard deviation of σ²_a is 141%; the 90% confidence interval half-width for w_s (the proportion of the stable component) is ±52.9 percentage points—so wide that any true value could be classified as either “stable-dominated” or “random-dominated.” To achieve ±10pp precision, k≥30 readers are needed; to reach ±5pp, k≥114 readers.
Researches on phenomena like Qi Gong external energy typically involve only two or three raters. Under this configuration, w_s is a number that is fundamentally impossible to estimate accurately, not merely “insufficient sample size.”
Literature corroborates this point. Kreiter and Wilson (PMID 26925540) explicitly state: under nested designs, systematic rater effects and “rater × subject” interaction effects are merged into a single confounded variance component, making them inseparable; separation is only possible via a fully crossed model. This is not a statistical technique issue but a structural limitation of the data itself. Svensson and Holm (PMID 7701145) provided a method in 1994 to decompose ordered categorical paired variation into systematic (determined by marginal totals) and random (relative rank variance) components—though the method exists, for two readers, the estimation precision of the systematic part is still capped by df. Sturman et al. (PMID 28587597) tracked examiner severity and found that even without feedback intervention, severity drifted noticeably over months; they themselves noted “this inherent instability makes the study of examiner training strategies complex.” In other words, the premise of “stable personal style” may not hold—it might simply be a variable drifting slowly enough to be mistaken for a constant.
Cut 2: Even if measured, the judgment direction is reversed.
This is the most critical finding of this round. The original reasoning assumed that the random component has zero expectation and does not systematically inflate TVD, so when randomness dominates, the deadband can be ignored.
This reasoning overlooks a sign issue: TVD is an even function of ε. Under three categories, TVD = max{|p₁−q₁|, |p₂−q₂|, |p₃−p₃|}, while our parameterization makes the subcategory difference exactly proportional to |ε_A − ε_B|. Thus, TVD depends only on the absolute difference of the two readers’ deviations, not the sign. One reader optimistic and one conservative (opposite signs, maximum difference) inflates TVD; both optimistic but to different degrees (same sign, non-zero difference) also inflate TVD. Therefore, “zero expectation” offers no protection here.
Quantitatively: under equal variance s, the stable component (two-point distribution ±s) has E|Δ| = s; the random component (Gaussian) has Δ ~ N(0, 2s²), so E|Δ| = √(4/π)·s ≈ 1.128s. The random component’s inflation capacity is actually 12.84% higher.
More critically, the hard upper bound: if both readers’ stable styles fall within |ε| ≤ ε, then |Δ| ≤ 2ε, and TVD always equals δ, never crossing the line. Scanning the full grid where |ε|≤ε for m̄=0.75/0.85/0.90 yields max(TVD) exactly equal to δ, with error on the order of 10⁻¹². Meanwhile, the random component follows a continuous distribution, so its tails always exceed any threshold: when m̄=0.85 and s=ε=8.33%, the crossing rate for pure stable |ε|=ε is 0.00%, while for pure random sd=ε it is 16.50% (from 50,000 simulations).
The conclusion is reversed: the stable component is safe; the random component is dangerous. The original plan’s claim that “random dominance ⇒ deadband warning downgraded to false alarm” is directionally wrong.
Cut 3: w_s is scale-invariant, but the ceiling burden is not.
This may be the most fundamental flaw. w_s = σ²_a/(σ²_a+σ²_r) is a ratio, so two datasets can have identical w_s yet differ in total variance by an order of magnitude. Empirical tests (m̄=0.85, 20,000 per group):
| σ²_a | σ²_r | w_s | E[TVD] − δ |
|---|---|---|---|
| 0.65 | 0.35 | 0.65 | +11.44pp |
| 0.13 | 0.07 | 0.65 | +4.02pp |
| 0.0325 | 0.0175 | 0.65 | +1.21pp |
w_s is identical, yet the ceiling burden differs by 9.5 times. Using w_s for judgment means applying a dimensionless ratio to a dimensional consequence—dimensionally inconsistent.
Furthermore, there is an internal contradiction: the original plan claimed “more repetitions average out random components, increasing w_s, so the deadband criterion should tighten.” But more repetitions actually reduce the burden. As r increases from 1 to 10, w_s rises from 35.0% to 84.3% (calculated with σ²_a=0.35, σ²_r=0.65), yet the pure random crossing rate drops from 16.34% to 0.00%. Thus, “w_s increasing” is itself a byproduct of “risk decreasing”; interpreting it as “should be more cautious” reverses the logic.
Proxy: Do not decompose; directly measure that difference.
Since TVD only consumes |Δ|=|ε_A−ε_B|, what needs judgment is essentially one number. The criterion should directly state: |Δ| ≤ 2ε* = δ/(2(1−m̄)). For m̄=0.85, this line is 16.67%; for m̄=0.75, it is 10.00%; for m̄=0.90, it is 25.00%.
This is a standard non-inferiority test, requiring sample size (one-sided α=.05, subcategory baseline proportion p=0.25):
| m̄ | Criterion line 2ε* | n (r=1) | n (r=2) | n (r=3) |
|---|---|---|---|---|
| 0.70 | 8.33% | 147 | 74 | 49 |
| 0.75 | 10.00% | 102 | 51 | 34 |
| 0.80 | 12.50% | 65 | 33 | 22 |
| 0.85 | 16.67% | 37 | 19 | 13 |
| 0.90 | 25.00% | 17 | 9 | 6 |
Comparison: decomposing w_s requires ≥40 readers to achieve 80% power, whereas directly measuring |Δ| needs only two readers and dozens to over a hundred cases. With the same research budget, one approach is feasible; the other is not.
Evidence and Limitations
Verified: Under three-category symmetric parameter families m_A=m̄±δ/2, TVD = max(three marginal differences absolute values), and the subcategory difference is proportional to |ε_A−ε_B|—this is pure algebra, with the closed-form solution exactly giving TVD=δ at ε=ε (verified pointwise across all three m̄ values, error <10⁻¹²). On the full grid where |ε|≤ε, max(TVD)=δ, confirming the hard upper bound for the stable component. Under equal variance, the random component’s E|Δ| exceeds the stable component by 12.84% (√(4/π)); simulation of 50,000 runs yields crossing rates of 16.50% vs 0.00%. When w_s is identical, the burden differs by 9.5 times; empirical tests confirm all three groups consistently. Required sample sizes were solved analytically via the non-inferiority formula.
Partially Verified: The degrees of freedom df=k−1 and corresponding confidence interval half-width for σ²_a follow standard variance component theory (normal approximation), without correction for small-sample exact distributions; w_s’s interval width uses the delta method’s first-order approximation, which incurs greater bias when k is very small. Literature only confirms the two general conclusions that “systematic and interaction components are inseparable under nested data” and “examiner severity itself drifts”; no study has performed the specific comparison of “stable vs. random components’ ability to inflate TVD.” The assumption that repeated ratings only suppress the random component is idealized—actual repetition may introduce memory effects, turning the second rating into a copy of the first, so the actual benefit of r is less than √r.
To Be Verified: The empirical distribution of |Δ| in real three-category adjudication data; whether the stable component is truly “stable” (Sturman et al.’s results suggest it may not be, requiring tracking across ≥4 time points to distinguish slow drift from constant); how much memory effects reduce the benefit of repeated ratings; how the line 2ε* shifts when m̄ and δ themselves carry sampling error (see #1405 dedicated thread).
Evidence Level: C
References: PMID 26925540, PMID 7701145, PMID 28587597, PMID 31169540L1 Network Status: The CORRESPONDENCE_NETWORK.md edge table ends at E-050 (Na Yin ↔ Frequency,
L1 | open); the original text of the modern evidence column states “no measurable source yet“; the network alignment indicator row states “0 edges beyond L2.” The three-domain node table contains no registration slots for “rating / rater / variance component / consistency” (full-table search for “rating / adjudication / consistency / marginal / bias / variance / noise / measurement” yields zero hits); the physical side’s “phase / rhythm” semantics refer to “two oscillators phase-locked,” not marginal distribution comparison. Therefore, this article’s evidence level is capped at C, and the topic has not yet entered the corresponding L1 network (registered to l1-gaps).
Further Reading
- Equal partition assumption does not bear load; δ is the floor: closed-form solution for three-category TVD deadband
- The ruler in the qualified zone cannot reach that line
- Resonance consciousness field research pool
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply