那些算出来的门槛为什么普遍差两三成,答案可能藏在没人报的东西里:已发表判读研究几乎只给κ与区间,不给两个读者各自的完整边际,于是「分歧摊在几条线上」这个量从未被观测过

Written by

in

Language

Evidence level: C

同一个偶然一致率、同一个真实分歧总量,仅因分歧落在哪几档不同,门槛就差两成上下——而决定这件事的关键量,恰恰是判读研究不报告的那一个。

一个被卡住的链条

判读一致性研究里有一条环环相扣的链:想定样本量门槛,先要算方差系数;想算方差系数,先要知道两位读者的分歧按什么形状摊在各个类别上。上一篇文章已经证明,这条链的最后一环不是可有可无的细节——同样的偶然一致率、同样的真实分歧总量,仅改变差落在哪几档,方差系数就能跨一点一四到一点三二倍,折算成样本量门槛是两成上下的差别;而实践里当唯一旋钮用的「偶然一致率单变量简化式」,在三百六十个格点的检验里只有不到两成能落到一成误差以内,最坏偏四点五倍。

问题在于:这个决定性的量——分歧的档位落点——在已发表文献里几乎找不到。主流报告格式是「kappa 等于某值、九成五区间多少」,再讲究一点补一个别的系数作对照;两位读者各自的分类别边际属于少数派做法,连观察一致率本身都常常缺席。也就是说,「文献里的门槛偏乐观还是偏保守」这个问题,今天在数据层面就无法回答——不是没人算过,是算它需要的原料从来没被系统地报出来。

为什么边际落点不是技术细节

有人会问:门槛差两成,多招两成人不就行了?这个直觉漏了两件事。第一,偏差没有固定方向。边际集中(分歧全压在一两档)与边际摊开(分歧均匀铺满各档)会把方差系数往相反方向拉,同样一份「看起来合理」的门槛,对不同研究可能一边过剩、一边严重不足。第二,单参数简化式的误差可以大到四点五倍,意味着照它算出来的门槛可能与真实需要差出一个数量级——这种误差不会靠「多留点余量」自然吸收。

方法学文献早有伏笔。kappa 系列系数对边际形态的敏感性自上世纪九十年代起就被反复指出:同样的边际结构能让系数在「看似很强」与「令人担忧」之间跳档,样本量公式的输入正是完整边际而非某个汇总值;二〇二二年一项蒙特卡洛研究进一步量化了边际不对称分布下各 kappa 型系数行为的漂移。链条早已写明,缺的只是文献层面那一手实测。

把「没被观测过的量」测出来

可行的做法是一次文献计量审计。筛选至少四十项报告了完整分类别边际(四格表或边际比例)的判读研究,逐项重建两位读者的边际与差落点,算出该研究的真实方差系数,再与它若按单参数简化式估出的系数相比,得到「文献门槛偏差」的比值分布。乳腺密度四档判读这类报告格式完整的研究是现成模板。

关键不在算,而在三条判据怎么切。若比值中位落在零点九到一点一,说明单参数简化在实践中大体够用,只是在极端构型上出错,处理办法是要求并报构型敏感性;若中位系统性偏离一超过两成,说明偏差有固定方向,那么「判读研究必须报告完整边际」应升级为强制规范,否则其门槛不可复核;若偏差在两个方向上各占一半,说明偏差不可先验修正,只能逐研究重算——门槛的「事前公示」目标就须降级为「事前给出区间」。

这一步为什么必须走在设计前面

我们自己的多池设计马上要用这些门槛。如果文献门槛的偏差结构摸不清,等于把设计手册架在一个没校准过的刻度尺上:门槛偏松,欠功效研究照常通过;门槛偏紧,样本量翻倍却没人知道为什么。先花一轮文献计量的成本,把「没人报的量」的偏差分布测出来,后面所有判读设计的样本量核算才有可追溯的依据——这正是这一系列文章反复强调的次序:判据的来历必须比判据本身更早被审。

证据与局限

已验证:kappa 类系数受边际分布支配(PMID 8501467 拆解偏倚、流行度与 kappa 三成分);边际不对称下各 kappa 型系数行为漂移已由蒙特卡洛量化(PMID 35124830);一致性研究的报告规范本身薄弱,样本量依据常缺(PMID 36123642)。

部分验证:完整边际是样本量公式的必要输入(PMID 22560852、11512137 的公式结构均以此为前提),但「文献门槛偏差」的实测分布尚无人做过,本文设计属待执行方案。

待验证:四十项以上研究的比值分布、三条判据的实际落点、以及「强制报边际」规范的可执行成本。注意:本文属统计方法学推演,与易学概念无直接对应,其价值在于为后续辨证判读类设计提供门槛校准的前置工具。

相关阅读

L1 网络状态:本主题(文献门槛偏差 × 方差系数边际形态依赖)在 CORRESPONDENCE_NETWORK 与 EDGE_TABLE_science-yixue 中均无对应边;最近邻为 E-060 标签前提边,其原文摘录:「KB-292(conflict):中医医师间诊断一致性在开放人群中『低(low)』,作者称为『令人担忧』(PMID 30383398)……标签信度,冲突(未裁决)」——该边处理标签信度本身,与本篇的门槛偏差审计不构成同一条边,故本文证据等级封顶 C。

证据等级: C
参考文献: PMID 8501467, PMID 35124830, PMID 36123642, PMID 22560852, PMID 11512137, PMID 37127220

证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Why calculated thresholds universally differ by 20–30%: the answer likely lies in what is never reported; published diagnostic studies almost exclusively provide κ and intervals…

Evidence level: C

The reason calculated thresholds are universally off by roughly 20–30% likely lies in the unreported data: published interpretation studies almost exclusively report κ (kappa) and confidence intervals, but never the complete marginal distributions for both readers. Consequently, the quantity “divergence spread across specific categories” has never been observed.

With the same chance agreement rate and the same total amount of real divergence, the threshold differs by about 20% simply because the divergence falls into different categories. The key quantity determining this difference is precisely the one not reported in interpretation studies.

A Stalled Chain

In consistency studies, there is an interlinked chain: to set the sample size threshold, one must first calculate the variance coefficient; to calculate the variance coefficient, one must know how the divergence between two readers is distributed across categories. As proven in the previous article, this final link is not a trivial detail—the same chance agreement rate and the same total amount of real divergence can result in variance coefficients ranging from 1.14 to 1.32 times, translating to a sample size threshold difference of about 20%. In practice, the “single-variable simplified formula for chance agreement,” used as the sole control knob, fails in nearly 80% of cases across 360 grid points, with the worst error reaching 4.5 times.

The problem is that this decisive quantity—the specific category distribution of divergence—is virtually absent in published literature. The standard reporting format is “kappa equals X, with a 95% confidence interval of Y,” sometimes supplemented by another coefficient for comparison; the individual marginal distributions for both readers belong to a minority approach, and even the observed agreement rate itself is often missing. Therefore, the question of whether “thresholds in the literature are overly optimistic or overly conservative” cannot be answered at the data level today—not because no one has calculated it, but because the raw materials required for such calculation have never been systematically reported.

Why Marginal Distribution Is Not a Technical Detail

One might ask: if the threshold differs by 20%, simply recruiting 20% more participants would suffice. This intuition overlooks two critical points. First, the bias does not have a fixed direction. When margins are concentrated (divergence compressed into one or two categories) versus when they are spread out (divergence evenly distributed across all categories), the variance coefficient is pulled in opposite directions. Thus, the same “seemingly reasonable” threshold may be excessively high for one study and severely insufficient for another. Second, the error of the single-parameter simplified formula can be as large as 4.5 times, meaning the calculated threshold could differ from the true requirement by an entire order of magnitude—an error that cannot be naturally absorbed by “adding extra margin.”

Methodological literature has long hinted at this. Since the 1990s, it has been repeatedly noted that kappa-series coefficients are highly sensitive to marginal distribution: the same marginal structure can cause a coefficient to jump between “appearing strong” and “concerning,” while the input for sample size formulas must be the complete marginal distribution rather than some aggregated value. A 2022 Monte Carlo study further quantified the drift in behavior of various kappa-type coefficients under asymmetric marginal distributions. The chain was already laid out; what was missing was the empirical measurement at the literature level.

Measuring the “Unobserved Quantity”

A feasible approach is a bibliometric audit. Select at least 40 studies reporting complete marginal distributions (four-fold tables or marginal proportions) and reconstruct the margins and divergence points for both readers, calculate the true variance coefficient for each study, then compare it with the coefficient estimated using the single-parameter simplified formula to derive the distribution of “literature threshold bias.” Studies on breast density with four categories provide ready-made templates.

The key is not the calculation itself, but how to apply three criteria:
– If the median ratio falls between 0.9 and 1.1, the single-parameter simplification is largely adequate in practice, though it fails under extreme configurations; the solution is to require and report configuration sensitivity.
– If the median systematically deviates by more than 20%, the bias has a fixed direction; thus, “interpretation studies must report complete margins” should become a mandatory standard, otherwise their thresholds cannot be verified.
– If the bias is split equally in both directions, it cannot be corrected a priori; each study must be recalculated individually—the goal of “pre-publication disclosure” must therefore downgrade to “pre-publication interval reporting.”

Why This Step Must Precede Design

Our own multi-pool design immediately requires these thresholds. If the bias structure of literature thresholds remains unclear, we are essentially building our design manual on an uncalibrated ruler: overly loose thresholds allow underpowered studies to pass; overly tight thresholds double the sample size without anyone knowing why. Spending one round of bibliometric cost to measure the bias distribution of “unreported quantities” ensures that all subsequent sample size calculations have traceable foundations—this is precisely the sequence repeatedly emphasized in this series of articles: the origin of the criteria must be audited before the criteria themselves.

Evidence and Limitations

Verified: Kappa-type coefficients are governed by marginal distribution (PMID 8501467 decomposes bias, prevalence, and kappa into three components); under asymmetric margins, drift in behavior of various kappa-type coefficients has been quantified via Monte Carlo (PMID 350124830); reporting standards for consistency studies are inherently weak, often lacking sample size basis (PMID 36123642).

Partially Verified: Complete margins are necessary inputs for sample size formulas (PMID 22560852, 11512137 both use this as a structural prerequisite), but the empirical distribution of “literature threshold bias” has never been measured; this article proposes a pending execution plan.

Pending Verification: The ratio distribution of over 40 studies, the actual landing points of the three criteria, and the implementation cost of mandating margin reporting. Note: This article belongs to statistical methodology derivation and does not directly correspond to Yijing Studies concepts; its value lies in providing a prerequisite tool for threshold calibration of future diagnostic interpretation designs.

Related Reading

L1 Network Status: This topic (literature threshold bias × variance coefficient marginal shape dependence) has no corresponding edge in CORRESPONDENCE_NETWORK or EDGE_TABLE_science-yixue; the nearest neighbor is the E-060 label prerequisite edge, whose original excerpt states: “KB-292 (conflict): diagnostic consistency among TCM practitioners in open populations is ‘low (low)’, authors call it ‘concerning’ (PMID 30383398) … label credibility, conflict (unresolved)”—this edge handles label credibility itself and does not constitute the same edge as this article’s threshold bias audit, hence this article’s evidence level caps at C.

Evidence Level: C
References: PMID 8501467, PMID 350124830, PMID 36123642, PMID 22560852, PMID 11512137, PMID 37127220

Evidence Level: A = directly supported by PubMed, B = scientific analogy / indirect evidence, C = traditional consensus / preliminary exploration


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *