Evidence level: C
「两位读者主档比例之差」这个量只在两人设计里存在。读者数一到三位以上,它有三个互不相同的替身,而这三个替身的零刻度能差到 2.65 倍——把两人设计的判据线搬过去,纯噪声会被判成「有稳定成分」,且读者越多判得越狠。
一、两人差是个只在两人世界里成立的量
前一篇把「δ 稳不稳」重写成「批次间方差 ÷ 含相关系数的二项方差」,零刻度定在 √(2−2ρ),判据线取 1.5 与 3.0。整套推导有一个被写死在开头的条件:读者恰好两位,且是固定的那两位。这个条件在真实判读研究里几乎不成立。三十位医师评三百多例病案的研究里,四百三十五对两两 κ 的中位只有 0.26,四分位区间 0.16 到 0.42,最小负 0.07、最大 0.62(PMID 22844445)——配对一多,配对本身就成了一种抽样。教育测量领域更直接:G 理论仿真给出的结论是,读者数少才是 ICC 偏倚与低效的主要来源,而不是题目数、也不是受试者数(PMID 33818118);而受试者数与读者数怎么分配,取决于读者方差与误差方差之比,比值不明时用相同数量最稳(PMID 16143966)。这些都是「加读者能提高精度」的证据,没有一条回答另一个问题:原来那条判据线,换了读者数还算不算数。
二、三个替身的零刻度:降的、降得更快的、几乎不动的
在等频三档、每批一百二十例、纯噪声的设定下,把三个替身的批间散布算到底(批次数放到四百,让估计收敛):两两配对取平均绝对差,读者数从二升到八,批间标准差从 3.88 个百分点降到 1.40,降了 64%;相对均值偏离(均方根)从 1.94 降到 1.11,本来就更窄、降幅也近似;边际极差从 3.88 只降到 3.72,几乎不动。极差与配对之比因此从 1.00 一路涨到 2.65。原题猜的是「极差期望随读者数上升、从二到八可涨近一倍」,实测不是近一倍,是 2.65 倍。原因是极差是极值型统计量:读者越多,最大最小两端各自往外走,多出来的读者主要贡献的是「更远的端点」,不是「更稳的中心」。这与一般化理论里读者方差被平均掉、误差被平均掉的直觉正相反——被平均掉的是中心,不是端点。
三、把 1.5 搬过去,纯噪声就有九成八被判成有结构
真正的灾难在搬运那一步。若沿用两人设计的地板去归一极差统计量,纯噪声下读者三位时已有 42.8% 越过 1.5 线;五位 89.0%;八位 97.8%,其中 22.2% 连 3.0 线都越了。也就是说,加读者本是为了让结论更可靠,实际效果是把「纯噪声」系统性地判成「有稳定成分」,而且读者越多判得越肯定。这个偏倚是单向的、有方向的、随设计改善而恶化的——最坏的一种错误。反过来,只要每个定义用它自己的零分布归一,三个定义在十二批次下越 1.5 线的概率都回到 1.1% 到 2.0%,名义水平恢复。所以问题不在多读者设计本身,而在判据线能不能跨定义搬。文献侧有同型证据:读者间异质性可以大到 eta 平方 0.40、超过被评者本身的 0.22,不做校正时假阴性率两成(PMID 40688984);评分者效应在多面设计里通常被报成「占比小于 5%」(PMID 26757931),但这个占比是方差占比,不是判据的零刻度。
四、还有第二道闸:批次数太少,连正确的线都会被越
归一化对了还不够。仍用配对定义、纯噪声,批次数四时批间散布的九十五分位是收敛值的一·六四倍,越 1.5 线的概率 8.5%;六批次 1.49 倍、4.9%;十二批次 1.35 倍、1.1%;二十四批次 1.24 倍、0.1%。也就是说「一点五」这条线在十二批次以下就已经被自身的抽样噪声吃掉一部分,四批次时几乎失效。这是一个原题没算的自由度:原题担心的是「读者数变了判据失效」,实测还叠了一层「批次少了判据自发越线」,两个方向都指向同一个结论——判据线不是常数,是(定义、读者数、批次数)三元组的函数。文献里读者数加到七位、G 系数从 0.86 升到 0.96,但五位已到 0.94,边际收益迅速递减(PMID 41181163),说明加读者不是免费的;而方差成分本身要六十位评分者、三十个刺激才估得稳(PMID 31898288),现实研究很少达到。
五、三个定义不是三重证据,分工结论倒是跨读者数稳定
最后一个反直觉点:配对、偏离、极差三个定义看起来像三条独立路径,实则共享同一批边际,高度相关。批乘读者交互漂移存在的条件下,三者「同时越线」的比例从读者两位的 2.4% 掉到八位的 0.5%——比单条路径还低,因为它们同向同源,一致通过不构成三重复现。另一方面,前一篇的分工结论跨读者数完全稳定:固定风格散布存在时,三个定义的越线率都停在噪声水平(1.2% 到 2.4%),散布型统计量对恒定风格偏移依旧零敏感;而位置型检验(批次均值之差相对其标准误)在同样条件下功效为 1.000。所以「散布管漂移、位置管偏移」这条分工不是两人设计的特例,换读者数照样成立;失效的只是那条被搬来搬去的判据线。
证据与局限
已验证:在阈值潜变量模型下,读者数由二增至八时,两两配对与相对均值偏离两种定义的纯噪声批间散布单调下降,边际极差几乎不动,两者之比由 1.00 升至 2.65;沿用两人设计地板归一极差统计量,读者八位时纯噪声越 1.5 线的概率达 97.8%;各定义用自身零分布归一后名义水平恢复至 1.1% 到 2.0%;批次数低于六时连正确归一化也会显著越线;固定风格偏移对三种散布型统计量零敏感,而位置型检验功效为 1.000。
部分验证:文献支持「读者数少是 ICC 偏倚主因」(PMID 33818118)、「读者与受试者数量的最优配比依赖方差比」(PMID 16143966)、「读者间异质性可超过被评者」(PMID 40688984)、「读者数边际收益递减」(PMID 41181163),但没有任何一篇给出跨读者数的判据线换算表,也没有一篇报过极差统计量的零分布。
待验证:一,本文的零分布来自等频三档加对称切点,非等频或偏斜切点下「读者数增大、极差零刻度上升」的斜率会不会变,未测;二,真实判读研究里读者是轮换而非全员评全批,稀疏连接下三种定义的方差还要再叠一层设计效应,本文按完全交叉算,属乐观估计;三,读者间存在系统分层(如资深与新手混编)时,极差会先被层间差撑大、再被层内波动填充,本文未分层。
相关阅读
证据等级: C
参考文献: PMID 22844445, PMID 33818118, PMID 16143966, PMID 40688984, PMID 26757931, PMID 41181163, PMID 31898288L1 网络状态:CORRESPONDENCE_NETWORK.md / EDGE_TABLE_science-yixue.md 中「读者」0 处、「rater」0 处、「判读」0 处、「极差」0 处、「轮换」0 处、「零分布」0 处、「kappa」0 处;「一致性」6 处、「信度」8 处全部落在 E-060 标签前提(五脏辨证作分组标签|前提|标签信度),该边状态为 conflict(未裁决):一侧为 PMID 30383398 判定开放人群中中医医师间诊断一致性「低(low)」、作者称「令人担忧」,另一侧为 KB-333/KB-334 指出「一致性 low」正落在加权系数精度最差区间、且 κ 型系数对序数/名义的敏感方向与 Gwet 一系相反,另有 PMID 31730403 等建议改用 Gwet AC2。该边只登记「标签信度够不够用」,未登记「换读者数后判据线的零刻度如何变」。据硬约束,本篇证据等级封顶 C,且不得写成已验证机制。
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
The entire floor is invalidated unless readers are fixed to those two: under multi-reader rotation, \”difference\” becomes \”dispersion,\” and the zero distribution of dispersion ha…
Evidence level: C
The quantity “difference between the two readers’ main file proportions” exists only in designs with exactly two readers. When the number of readers reaches three or more, this quantity has three distinct proxies, and the zero scale of these three proxies can differ by up to 2.65 times—transplanting the criterion line from the two-reader design would cause pure noise to be judged as “having stable components,” and the more readers, the harsher the judgment.
I. The Two-Reader Difference is a Quantity That Holds Only in the Two-Reader World
The previous article rewrote “whether δ is stable” as “batch variance ÷ binomial variance containing correlation coefficients,” setting the zero scale at √(2−2ρ) and the criterion line at 1.5 and 3.0. The entire derivation rests on one condition hard-coded at the start: exactly two readers, and they must be fixed. This condition is almost never met in real-world judgment studies. In a study of 30 physicians rating over 300 cases, the median κ for 435 pairs was only 0.26, with interquartile range 0.16 to 0.42, minimum negative 0.07 and maximum 0.62 (PMID 22844445)—once pairing increases, the pair itself becomes a form of sampling. Educational measurement is even more direct: G-theory simulations conclude that fewer readers are the main source of ICC bias and inefficiency, not the number of items or subjects (PMID 33818118); and how to allocate subjects versus readers depends on the ratio of reader variance to error variance, so when this ratio is unknown, equal numbers are most stable (PMID 16143966). All these constitute evidence that “adding readers improves precision,” but none answers the other question: once the criterion line is transplanted, does it still hold?
II. Three Proxies with Zero Scales: One Drops, One Drops Faster, One Almost Stays Still
Under the equal-frequency three-tier, 120 cases per batch, pure noise setup, we computed the batch-to-batch spread of the three proxies to convergence (setting batches to 400): taking pairwise averages of absolute differences, as readers increase from two to eight, the batch standard deviation drops from 3.88 percentage points to 1.40—a 64% reduction; the relative mean deviation (root mean square) falls from 1.94 to 1.11, which was already narrower and the drop is approximately proportional; the marginal range shrinks only from 3.88 to 3.72, almost unchanged. Consequently, the ratio of range to pairwise average rises from 1.00 all the way to 2.65. The original conjecture was that “the expected range increases with reader count, nearly doubling from two to eight,” but the actual result is not nearly double—it is 2.65 times. This is because range is an extreme-value statistic: as readers increase, the maximum and minimum each move outward further; the extra readers mainly contribute “more distant endpoints,” not “more stable centers.” This contradicts the general theory intuition that reader variance gets averaged out and error gets averaged out—what gets averaged is the center, not the endpoints.
III. Transplanting 1.5 Leads to 98% of Pure Noise Being Judged as Having Structure
The real disaster occurs at the transplantation step. If we continue using the two-reader design floor to normalize the range statistic, under pure noise with three readers, 42.8% already exceed the 1.5 line; five readers: 89.0%; eight readers: 97.8%, of which 22.2% even cross the 3.0 line. In other words, adding readers was intended to make conclusions more reliable, but the actual effect is to systematically judge “pure noise” as “having stable components,” and the more readers, the more certain the judgment. This bias is unidirectional, directional, and worsens with design improvement—the worst kind of error. Conversely, if each definition uses its own zero distribution for normalization, then under twelve batches, the probability of exceeding the 1.5 line returns to 1.1% to 2.0%, restoring the nominal level. Thus the problem lies not in the multi-reader design itself, but whether the criterion line can be transplanted across definitions. Literature provides homologous evidence: reader heterogeneity can reach eta-squared 0.40, exceeding the subject’s own 0.22, and without correction, the false negative rate is 20% (PMID 40688984); rater effects in multifaceted designs are usually reported as “accounting for less than 5%” (PMID 26757931), but this proportion refers to variance contribution, not the zero scale of the criterion.
IV. There Is a Second Gate: Too Few Batches, Even the Correct Line Gets Exceeded
Normalization alone is insufficient. Still using the pairwise definition under pure noise, with four batches, the 95th percentile of batch-to-batch spread is 1.64 times the convergence value, giving an 8.5% probability of exceeding the 1.5 line; six batches: 1.49 times, 4.9%; twelve batches: 1.35 times, 1.1%; twenty-four batches: 1.24 times, 0.1%. In other words, the “1.5” line is already partially consumed by its own sampling noise below twelve batches, and almost completely ineffective at four batches. This is a degree of freedom not calculated in the original problem: the original worried about “reader count changes causing criterion failure,” but the actual result adds another layer—”fewer batches cause the criterion to spontaneously cross the line”—both directions pointing to the same conclusion: the criterion line is not constant, but a function of the triplet (definition, reader count, batch count). Literature shows that adding readers to seven increases the G coefficient from 0.86 to 0.96, but five already reaches 0.94, so marginal gains rapidly diminish (PMID 41181163), indicating that adding readers is not free; and variance components themselves require 60 raters and 30 stimuli to estimate stably (PMID 31898288), which is rarely achieved in real studies.
V. The Three Definitions Are Not Triple Evidence, but Their Division of Labor Is Stable Across Reader Counts
The final counterintuitive point: the pairwise, deviation, and range definitions appear to be three independent paths, yet they share the same marginal and are highly correlated. Under the condition that batch-by-reader interaction drift exists, the proportion of “simultaneous crossing” drops from 2.4% with two readers to 0.5% with eight—lower than any single path, because they originate from the same source and pass together does not constitute triple recurrence. On the other hand, the division-of-labor conclusion from the previous article remains completely stable across reader counts: when fixed style drift exists, the crossing rates of all three definitions stay at noise levels (1.2% to 2.4%), and the spread-type statistics remain zero-sensitive to constant style shifts; meanwhile, the location-type test (batch mean difference relative to its standard error) achieves power of 1.000 under the same conditions. Thus, “spread manages drift, location manages shift” is not a special case of the two-reader design—it holds regardless of reader count; what fails is only that transplanted criterion line.
Evidence and Limitations
Verified: Under the threshold latent variable model, as reader count increases from two to eight, the batch-to-batch spread of both pairwise and relative mean deviation definitions under pure noise monotonically decreases, while the marginal range remains almost unchanged; their ratio rises from 1.00 to 2.65; normalizing the range statistic using the two-reader design floor leads to 97.8% of pure noise exceeding the 1.5 line; after each definition is normalized using its own zero distribution, nominal levels recover to 1.1% to 2.0%; with fewer than six batches, even correct normalization significantly crosses the line; fixed style drift is zero-sensitive to all three spread-type statistics, while the location-type test achieves power of 1.000.
Partially Verified: Literature supports “few readers are the main source of ICC bias” (PMID 33818118), “optimal ratio of readers to subjects depends on variance ratio” (PMID 16143966), “reader heterogeneity can exceed subject heterogeneity” (PMID 40688984), and “marginal gain of readers diminishes” (PMID 41181163), but no study provides a criterion-line conversion table across reader counts, nor any reports the zero distribution of the range statistic.
To Be Verified: First, whether the slope of “increasing reader count raises the range zero scale” changes under unequal-frequency or asymmetric cutpoints, since this study uses equal-frequency three tiers with symmetric cutpoints; second, in real judgment studies readers rotate rather than all rate all batches, so under sparse connectivity the variance of the three definitions must also layer design effects, whereas this study assumes complete crossing and thus represents an optimistic estimate; third, when systematic stratification exists among readers (e.g., mixing experts and novices), the range is first inflated by inter-layer differences and then filled by intra-layer fluctuations, a scenario not addressed here.
Related Reading
- If the floor and the molecule share origin, then the ruler itself points to 1
- The scale actually measures nothing but “stability or not”
- Resonant consciousness field: why measurement consistency is the entry ticket for this edge
Evidence Level: C
References: PMID 22844445, PMID 33818118, PMID 16143966, PMID 40688984, PMID 26757931, PMID 41181163, PMID 31898288L1 Network Status: In CORRESPONDENCE_NETWORK.md / EDGE_TABLE_science-yixue.md, the “reader” 0, “rater” 0, “judgment” 0, “range” 0, “rotation” 0, “zero distribution” 0, and “kappa” 0 entries; the “consistency” 6 and “reliability” 8 entries all fall under the E-060 label premise (organ differentiation as grouping label | premise | label reliability), and this edge status is conflict (unresolved): one side cites PMID 30383398, which judges diagnostic consistency among TCM physicians in the open population as “low (low),” with authors calling it “concerning”; the other side cites KB-333/KB-334, which indicates that “consistency low” falls precisely in the worst precision interval of weighted coefficient accuracy, and the ordinal/nominal sensitivity direction of κ-type coefficients opposes that of the Gwet series, with PMID 31730403 and others recommending switching to Gwet AC2. This edge only records “whether label reliability is sufficient,” but does not record “how the zero scale of the criterion line changes after switching reader counts.” Under hard constraints, this article’s evidence level caps at C and cannot be written as a verified mechanism.
Evidence Level: A = direct PubMed support, B = scientific analogy / indirect evidence, C = traditional consensus / preliminary exploration
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply