Evidence level: C
1388 说「ρ>0.5 就该亮红牌」。把折扣的收益和代价放进同一个决策框架算下来,ρ 越高,打折越划算,不是越危险。那条红线不是错的线,是画错了变量——它真正在测的是「你的检验还有没有功效」,不是「你会不会判错」。
正文
一、上一轮那句话是我拍的,而且拍反了
1388 在第三节末尾写了这么一句:ρ>0.5 时除了打折,还要额外声明「可比性存疑」。理由很顺:折价过半就值得警惕。
但「折价过半」和「危险」是两件事。折价过半说明门槛降了一半,门槛降低既可能是「把该放行的放行了」(收益),也可能是「把不该放行的也放行了」(代价)。到底哪头大,取决于被比较的两份判读里有多大比例真有差——这个比例在一致性研究里没人报过,我也没算过就下了判断。
现在把它算清。设定很干净:T = κ̂₁ − κ̂₂ 近似正态,真差为零时 T ~ N(0, 2(1−ρ)SE²)。两条规则:不打折用阈值 1.96·√2·SE,打折用 1.96·√(2−2ρ)·SE。两者差一个 √(1−ρ)。
先看一个让人意外的事实:如果 ρ 是精确已知的,打折后的假阳性率恒等于 5.000%——不是 4.4%,不是 2.1%,是精确的 5.000%,ρ 从 0 走到 0.95 一动不动。因为打折用的本来就是正确的标准误,1.96 这个系数在正确标准误上就是 5%。
也就是说,打折从来不会制造假阳性。真正在制造假阳性的是不打折——但方向是反的:不打折让阈值变得过严,假阳性率从 5% 掉到 ρ=0.5 时的 0.56%、ρ=0.9 时的 0.0002%。
二、收益与代价的全表
固定真差 = 1 个独立 SE 单位(下面再谈为什么这个点最关键):
| ρ | 打折扣功效 | 不打折功效 | 收益 | 假阳性代价 | 临界先验 π* |
|---|---|---|---|---|---|
| 0.20 | 20.09% | 14.20% | +5.89pp | +2.16pp | 0.268 |
| 0.30 | 22.30% | 12.58% | +9.72pp | +3.08pp | 0.241 |
| 0.50 | 29.30% | 8.73% | +20.57pp | +4.44pp | 0.178 |
| 0.70 | 44.67% | 3.98% | +40.69pp | +4.97pp | 0.109 |
| 0.90 | 88.54% | 0.12% | +88.42pp | +5.00pp | 0.054 |
代价那一列封顶在 5pp——因为它就是「从不打折的 5% 涨回正确的 5%」。收益那列没有上限,ρ 越大涨得越凶。
π* 的定义是:被比较的判读里真有差的比例超过多少,打折才净赚。ρ=0.5 时这个门槛是 0.178。也就是说,只要「三个比较里有一个真的有差」,打折就划算。ρ=0.9 时门槛降到 0.054——十八个比较里有一个真有差就够本。
所以 ρ 越高,打折越划算。红线该画在低 ρ,不在高 ρ。
三、π* 的形状:为什么这不可能是单调的
如果 π* 随 ρ 单调下降,那「ρ 越大越该打折」就是普遍规律。它不是——它在 z 上有个谷,然后反弹。
| 真差 z | ρ=0.3 | ρ=0.5 | ρ=0.7 | ρ=0.9 |
|---|---|---|---|---|
| 0.5 | 0.383 | 0.332 | 0.254 | 0.124 |
| 1.0 | 0.241 | 0.178 | 0.109 | 0.054 |
| 1.5 | 0.178 | 0.127 | 0.079 | 0.051 |
| 2.0 | 0.173 | 0.135 | 0.104 | 0.100 |
| 2.5 | 0.223 | 0.212 | 0.240 | 0.533 |
| 3.0 | 0.360 | 0.428 | 0.635 | 0.990 |
π* 的最低点落在 z≈1.3–1.8(ρ 越大越往前挪)。越过 z≈2.5 之后急升,ρ=0.9 在 z=3 时到 0.99——几乎没有任何先验能让打折划算。
机制很直白:折扣把阈值从 1.96√2 拉到 1.96√(2−2ρ),这个平移量在高 ρ 时极大(ρ=0.9 时阈值只剩三分之一)。当真差本身已经大到能压过原阈值时,打折不再提供收益,只提供额外的假阳性。 收益是对「差一点就够」的人群起作用的——一旦真差本来就够,折扣就纯亏。
这就给出一条可直接用的操作规则:先估 Δ/D_min 这个比值,比值在 0.7–2.5 之间才谈打折;超过 2.5 一律不打,打了纯亏。
四、这条红线的真身:它是个功效告警
把 ρ=0.5 这个点单独拎出来看,它到底在说什么?
不打折的功效在 ρ=0.5 时是 8.73%。八个百分点的功效,意味着一份真有中等差值的比较,九成会被判成「不可辨」。ρ=0.7 时功效 3.98%,ρ=0.9 时 0.12%——这个检验已经死了。
所以 ρ>0.5 真正标记的是「你的检验已经失去分辨能力」,不是「你的结果不可信」。这两个警告要求的动作相反:前者要求必须打折(或者干脆承认测不出来),后者要求不要打折。#1388 把前者写成了后者。
而 ρ 在现实中集中在什么位置?#1388 那批仿真里,最常见的场景(同批受试者、换一组读者对)ρ 中位数只有 0.111,36 格里 5 格为负。ρ 天然就低,高 ρ 只出现在「同一对读者、隔一段时间再判一遍」这种特定设计里。 换句话说,这条红线在实际数据上几乎从不触发——它是一道针对罕见情形的、方向还写反了的门。
五、还有个更实在的问题:ρ 根本估不准
上面全部计算都假定 ρ 已知。它是估的,σ(ρ̂)≈(1−ρ²)/√n。
用 1.645 个标准误的高估(双侧检验里足以把 FPR 推过 10% 的量级)反解所需精度:
| 真 ρ | 允许 σ(ρ̂) | 自估所需 n |
|---|---|---|
| 0.30 | ≤0.1258 | ≥53 |
| 0.50 | ≤0.0898 | ≥70 |
| 0.70 | ≤0.0539 | ≥90 |
| 0.90 | ≤0.0180 | ≥112 |
n=60 时实测:ρ=0.5 的 FPR 均值 5.57%、95 分位 10.54%、6.2% 的批次会冲破 10%;ρ=0.9 时均值 5.98%、95 分位 13.31%、11.7% 的批次冲破 10%。#1388 表里那个「ρ=0.256 时 FPR 从 2.10% 涨到 4.40%」,落在 5.57% 均值附近的抽样波动范围内——它不是 ρ 造成的,是有限样本造成的。上一篇把抽样噪声读成了机制。
六、没有先验怎么办:用「对半赌」这条退路
π* 里那个 π 是真差先验,实务中确实不知道。退一步取 π=0.5(完全无知,值得与不值得打平),得到的边界极其宽松:
| ρ | 打折划算的 z 区间(π=0.5) |
|---|---|
| 0.2 | 0.05 – 3.38 |
| 0.5 | 0.05 – 3.11 |
| 0.9 | 0.05 – 2.48 |
ρ=0.5 时,只要 Δ < 3.11·D_min,打折就值得。 这是一条比 π* 粗糙但不用估先验的规则,覆盖面比绝大多数实际研究都宽。
取 π=0.25(「四个里有一个真有差」,判读研究里偏保守的合理值)时,区间收窄成 ρ=0.5 的 [0.72, 2.62]。下界 0.72 这条线值得记住:当真差小到连 D_min 的七成都不到时,打折也救不回来——此时问题不是门槛,是效应本身就太细。
证据与局限
已验证:① 精确已知 ρ 时,折扣规则的假阳性率恒为 5.000%,与 ρ 无关(ρ∈{0,0.3,0.5,0.7,0.9,0.95} 逐点验算);② 不打折的假阳性率随 ρ 单调塌陷(5.00%→0.56%→0.0002%),即折扣在修的是「过严」而非「过松」;③ π* 在 z 上非单调,谷底落在 z≈1.3–1.8,z>2.5 后急升,这是折扣阈值平移量与真实效应竞争的必然结果,不是数值巧合;④ 所需 ρ̂ 精度与 n 的换算式由 (1−ρ²)/√n 直接反解。
部分验证:① z 的定义用了「以独立 SE 为单位」,而真实研究的 SE 依赖 m、δ、n 一组参数,把 z 单一化是简化;② π* 曲面是正态近似下算的,小样本或高 ρ 时 T 的分布偏离正态(这正是第五节 FPR 分布右尾变厚的来源);③ ρ̂ 的方差近似式 (1−ρ²)/√n 假定双变量正态,实际 kappa 的联合分布未必满足。
待验证:① 真差先验 π 在任何真实一致性研究里都没被估过,本文所有「划算/不划算」的判决都压在它上面,而它恰好是唯一没有数据支撑的量;② 「Δ/D_min 先算再决定折不打折」这条流程要求先知道 D_min,而 D_min 里的 δ、ρ 两个输入本身都带误差,误差传播从未算过;③ 本文假定两位读者的 kappa 是同一批受试者上的成对比较,若研究里比较的是「两组不同受试者、不同读者」的结构,ρ 的定义直接失效、整张表不适用;④ 折扣与不折扣的判决用的是同一个 5% 名义水平,若研究采用更严的水平(如 1%),π* 曲面整体要重算。
L1 网络状态:CORRESPONDENCE_NETWORK.md 边表止于 E-050「六十甲子纳音 | 对应 | 60 个频率」(层级 L1、状态 open,现代证据栏原文「尚无任何可测来源」);全文件检索「判读 / 信度 / 一致率 / 相依 / 净收益 / 阈值 / 假阳性」仅命中第 79 行层级定义与第 159 行「L2 以上边数 | 0」。该主题尚未进入 L1 对应网络。
相关阅读
证据等级: C
参考文献: PMID 17099194, PMID 26810254, PMID 25155798, PMID 40405411, PMID 38841381, PMID 36872574, PMID 41239778, PMID 36123642, PMID 18430244, PMID 3337089
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
That ρ=0.5 red line, which was drawn not as danger but as gain: an intuition never derived before, once calculated, completely reversed the direction
Evidence level: C
1388 states that “ρ > 0.5 should trigger a red flag.” When the discounted benefit and cost are calculated within the same decision framework, higher ρ makes discounting more advantageous, not more dangerous. That red line is not the wrong line; it misidentifies the variable—it actually measures “whether your test retains power,” not “whether you will make a false negative.”
Body
1. That previous statement was mine, and I got it backwards
1388 wrote at the end of section three: “When ρ > 0.5, besides discounting, one must additionally declare ‘comparability is questionable’.” The reasoning seems straightforward: discounting beyond half is worth warning about.
However, “discounting beyond half” and “danger” are two distinct matters. Discounting beyond half indicates the threshold has halved. A lowered threshold can either mean “allowing those who should be allowed” (gain) or “allowing those who should not be allowed” (cost). Which side dominates depends on the proportion of comparisons that truly differ—a ratio never reported in consistency studies, and which I did not calculate before issuing the judgment.
Now let us calculate it precisely. The setup is clean: T = κ̂₁ − κ̂₂ approximates a normal distribution; when the true difference is zero, T ~ N(0, 2(1−ρ)SE²). Two rules: no discounting uses threshold 1.96·√2·SE, discounting uses 1.96·√(2−2ρ)·SE. The difference between them is √(1−ρ).
First, consider a surprising fact: if ρ is precisely known, the false positive rate after discounting is constant at 5.000%—not 4.4%, not 2.1%, but exactly 5.000%, regardless of ρ moving from 0 to 0.95. This is because discounting uses the correct standard error, and the coefficient 1.96 applied to the correct standard error yields exactly 5%.
Thus, discounting never creates false positives. What creates false positives is not discounting—but the direction is reversed: not discounting makes the threshold overly strict, driving the false positive rate from 5% down to 0.56% when ρ=0.5, and down to 0.0002% when ρ=0.9.
2. Full table of gain and cost
Fixed true difference = 1 independent SE unit (we discuss why this point is critical below):
| ρ | Discounted power | Undiscounted power | Gain | False positive cost | Critical prior π* |
|---|---|---|---|---|---|
| 0.20 | 20.09% | 14.20% | +5.89pp | +2.16pp | 0.268 |
| 0.30 | 22.30% | 12.58% | +9.72pp | +3.08pp | 0.241 |
| 0.50 | 29.30% | 8.73% | +20.57pp | +4.44pp | 0.178 |
| 0.70 | 44.67% | 3.98% | +40.69pp | +4.97pp | 0.109 |
| 0.90 | 88.54% | 0.12% | +88.42pp | +5.00pp | 0.054 |
The cost column caps at 5pp—because it represents “the 5% of never discounting returning to the correct 5%.” The gain column has no upper limit; the higher ρ, the more it rises.
The definition of π* is: above what proportion of true differences must exist in the compared judgments for discounting to be net profitable. When ρ=0.5, this threshold is 0.178. That means as long as “one out of three comparisons truly differs,” discounting is worthwhile. When ρ=0.9, the threshold drops to 0.054—one out of eighteen comparisons truly differing is enough.
Therefore, the higher ρ, the more advantageous discounting becomes. The red line should be drawn at low ρ, not high ρ.
3. The shape of π*: why this cannot be monotonic
If π* decreased monotonically with ρ, then “the higher ρ, the more one should discount” would be a universal rule. It is not—it has a valley in z, then rebounds.
| True difference z | ρ=0.3 | ρ=0.5 | ρ=0.7 | ρ=0.9 |
|---|---|---|---|---|
| 0.5 | 0.383 | 0.332 | 0.254 | 0.124 |
| 1.0 | 0.241 | 0.178 | 0.109 | 0.054 |
| 1.5 | 0.178 | 0.127 | 0.079 | 0.051 |
| 2.0 | 0.173 | 0.135 | 0.104 | 0.100 |
| 2.5 | 0.223 | 0.212 | 0.240 | 0.533 |
| 3.0 | 0.360 | 0.428 | 0.635 | 0.990 |
The minimum of π* occurs at z≈1.3–1.8 (the valley shifts forward as ρ increases). Beyond z≈2.5 it rises sharply; at ρ=0.9 and z=3, π*=0.99—almost no prior can make discounting worthwhile.
The mechanism is straightforward: discounting shifts the threshold from 1.96√2 to 1.96√(2−2ρ); this shift is large at high ρ (at ρ=0.9 the threshold shrinks to one-third). When the true difference itself is already large enough to surpass the original threshold, discounting no longer provides gain; it only adds extra false positives. Gain applies to the population “just barely sufficient”—once the true difference is already sufficient, discounting is purely a loss.
This yields an immediately usable operational rule: first estimate the ratio Δ/D_min; discounting is discussed only if this ratio lies between 0.7 and 2.5; above 2.5, never discount, as discounting is purely a loss.
4. The true identity of this red line: it is a power warning
Isolating the point ρ=0.5, what does it signify?
The power of not discounting at ρ=0.5 is 8.73%. Eight percent power means that a comparison with a medium true difference will be judged “indistinguishable” in ninety percent of cases. At ρ=0.7, power is 3.98%; at ρ=0.9, it is 0.12%—this test is dead.
Therefore, ρ > 0.5 truly marks “your test has lost its ability to distinguish,” not “your results are untrustworthy.” These two warnings demand opposite actions: the former requires must discount (or simply admit it cannot be measured), while the latter requires do not discount. #1388 wrote the former as the latter.
Where does ρ naturally lie in reality? In #1388’s simulations, the most common scenario (same subjects, different readers) has a median ρ of only 0.111, with 5 out of 36 cells negative. ρ is naturally low; high ρ appears only in specific designs such as “the same pair of readers, re-evaluated after some time.” In other words, this red line is almost never triggered by actual data—it is a door aimed at rare cases, with its direction written backwards.
5. A more practical problem: ρ cannot be estimated accurately
All calculations above assume ρ is known. It is estimated, with σ(ρ̂)≈(1−ρ²)/√n.
Solving for required precision using 1.645 standard errors (enough to push the false positive rate over 10% in a two-sided test):
| True ρ | Allowed σ(ρ̂) | Required n for self-estimation |
|---|---|---|
| 0.30 | ≤0.1258 | ≥53 |
| 0.50 | ≤0.0898 | ≥70 |
| 0.70 | ≤0.0539 | ≥90 |
| 0.90 | ≤0.0180 | ≥112 |
With n=60, actual results show: for ρ=0.5, the mean false positive rate is 5.57%, the 95th percentile is 10.54%, and 6.2% of batches exceed 10%; for ρ=0.9, the mean is 5.98%, the 95th percentile is 13.31%, and 11.7% of batches exceed 10%. The entry in #1388’s table stating “when ρ=0.256, the false positive rate rises from 2.10% to 4.40%” falls within the sampling fluctuation around the 5.57% mean—it is not caused by ρ, but by finite sample size. The previous article misread sampling noise as mechanism.
6. What if there is no prior: using the “half-or-all bet” fallback
The π in π* is the true difference prior, which is indeed unknown in practice. Taking a step back and setting π=0.5 (complete ignorance, where worth discounting equals not worth discounting) yields an extremely loose boundary:
| ρ | z interval where discounting is worthwhile (π=0.5) |
|---|---|
| 0.2 | 0.05 – 3.38 |
| 0.5 | 0.05 – 3.11 |
| 0.9 | 0.05 – 2.48 |
When ρ=0.5, discounting is worthwhile as long as Δ < 3.11·D_min. This is a rule coarser than π* but does not require estimating the prior, and its coverage is wider than most actual studies.
Taking π=0.25 (“one out of four truly differs,” a conservative reasonable value in judgment studies) narrows the interval to [0.72, 2.62] for ρ=0.5. The lower bound of 0.72 is worth remembering: when the true difference is less than seventy percent of D_min, discounting cannot save it—here the problem is not the threshold, but the effect itself is too fine.
Evidence and Limitations
Verified: ① When ρ is precisely known, the false positive rate of the discounting rule is constant at 5.000%, independent of ρ (verified point-by-point for ρ∈{0, 0.3, 0.5, 0.7, 0.9, 0.95}); ② the false positive rate of not discounting collapses monotonically with ρ (5.00% → 0.56% → 0.0002%), meaning discounting fixes “overly strict” rather than “overly loose”; ③ π* is non-monotonic in z, with the valley at z≈1.3–1.8 and a sharp rise beyond z>2.5, which is the inevitable result of competition between the discount threshold shift and true effect size, not numerical coincidence; ④ the conversion formula for required ρ̂ precision and n is directly derived from (1−ρ²)/√n.
Partially verified: ① the definition of z uses “independent SE units,” while real-world study SE depends on parameters m, δ, n; simplifying z to a single unit is an approximation; ② the π* surface is calculated under normal approximation; in small samples or high ρ, the distribution of T deviates from normal (this is precisely the source of the thick right tail in section five’s FPR distribution); ③ the variance approximation for ρ̂, (1−ρ²)/√n, assumes bivariate normality; the joint distribution of actual kappa may not satisfy this.
To be verified: ① the true difference prior π has never been estimated in any real consistency study; all judgments of “profitable/not profitable” in this paper rest on it, yet it is the only quantity without data support; ② the procedure “first compute Δ/D_min, then decide whether to discount” requires knowing D_min in advance, but the inputs δ and ρ inside D_min themselves carry error, and error propagation has not been calculated; ③ this paper assumes the comparison of two readers’ kappa is a paired comparison on the same subjects; if the study compares “two groups of different subjects and different readers,” the definition of ρ fails directly, rendering the entire table inapplicable; ④ the decision between discounting and not discounting uses the same nominal 5% level; if the study adopts a stricter level (e.g., 1%), the π* surface must be recalculated entirely.
L1 Network Status: The edge list in CORRESPONDENCE_NETWORK.md stops at E-050 “Sixty Jiazi Na Yin | Correspondence | 60 frequencies” (level L1, status open, original text in modern evidence column states “no measurable source yet“); searching the entire file for “judgment / reliability / consistency / dependence / net gain / threshold / false positive” hits only row 79 (level definition) and row 159 (“L2 and above edge count | 0”). This topic has not yet entered the L1 correspondence network.
Related Reading
- 2.772 This coefficient is correct; the error is your belief that it needs correction
- Two corrections are neither additive nor multiplicative
- The ceiling is inherently a subtraction
- Pool ① Resonance Consciousness Field
Evidence Level: C
References: PMID 17099194, PMID 26810254, PMID 25155798, PMID 40405411, PMID 38841381, PMID 36872574, PMID 41239778, PMID 36123642, PMID 18430244, PMID 3337089
Evidence Level: A = direct PubMed support; B = scientific analogy / indirect evidence; C = traditional consensus / preliminary exploration.
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply