Evidence level: C
在金标准信度只有 kappa=0.28 的前提下,”三质判别 ΔAUC 头对头”能否裁决,取决于两件事的乘积:标签噪声对 ΔAUC 的稀释幅度,和样本量对 DeLong 检验标准误的收缩速度。本文用模拟把”可检出 ΔAUC”随样本量的收缩曲线画出来,预先划定可裁决区与不可裁决区。
正文
围绕三质判别力的 ΔAUC 头对头比较,此前一系列裁决设计(编码方式、分档粒度、噪声注入、插补策略)都默认了一个前提:只要效应真存在,实验就能把它测出来。这个前提在金标准信度高的领域大体成立,但在体质标签这条线上必须先算账——既往评估里金标准一致性已报出 kappa=0.28 量级的低位,标签本身就有相当比例与真类别不一致。两个后果随之而来:第一,标签错分会把观测 AUC 系统性压低且变形,”不完美金标准偏倚”在诊断方法学里早有正式命名与估计框架(PMID 16011710);第二,标签噪声会稀释组间判别差异,使真 ΔAUC 在观测面上缩水。近年方法学进展给出了部分补救:在参照标准不完美的条件下,两个 AUC 差值的符号仍可识别,可据此做相对优劣检验(PMID 38994641);密度比模型下的似然方法也能对 ROC/AUC 做校正估计(PMID 41351166)。但任何校正都改变不了另一条更硬的约束——统计功效。
ΔAUC 检验的标准误由三个量决定:两组各自的 AUC 方差与二者的协方差,相关 ROC 设计下后者不可忽略。相关曲线比较的非参数框架正是 DeLong 协方差方法(PMID 3203132),样本量估算则有按双正态 ROC 指标推导的经典公式(PMID 9249923)。关键在于:ΔAUC 检出下限大致按 1/√n 收缩,且样本量预估对两个相关参数高度敏感,假设稍偏,所需样本量可以差出数倍,只能给出上下界(PMID 20160839)。把这条曲线和 kappa=0.28 的标签稀释叠加,就得到本文要标定的东西:同一个真 ΔAUC,在不同样本量格点上是从”可检出”掉进”不可裁决”的。
具体模拟设计分三层。第一层扫真 ΔAUC 梯度:0.02、0.04、0.06、0.08、0.10 五档,覆盖从”细效应”到”粗效应”的跨度。第二层扫金标准 kappa 梯度:0.2、0.4、0.6、0.8 四档,其中 0.2 档代表本线现状(0.28 附近的更保守邻域),0.8 档代表标签质量上限。第三层扫样本量:每组 100、200、400、800、1600 五档。三层组合共 100 个格点,每格点跑 1000 次重复模拟:先按 kappa 档把真标签翻错成观测标签,再在观测标签上算两模型 AUC 与 DeLong 相关检验的功效。判读规则预注册两条:某格点功效 ≥0.80 记”可裁决区”;功效 <0.50 记”不可裁决区”,该区域内任何”没测出差异”的结果一律不得解读为阴性证据。介于两者之间为灰色带,须加倍样本量或合并多批数据后再判。
三条实施细节必须写进方案。其一,标签翻转率必须由 kappa 反推并在模拟中显式实现,不能只把 AUC 打折——错分对 AUC 的压缩是非线性的,且依赖疾病占比。其二,DeLong 检验须用同批受试者的配对设计,协方差项才有效;用两批独立样本套配对公式会低估方差,虚增功效。其三,参照标准不完美时若要下”孰优孰劣”结论,应改用可识别性已证明的 AUC 差值符号检验或似然校正框架(PMID 38994641;PMID 41351166),并把两种口径的结果并列报告——若”观测标签口径”与”校正口径”结论相反,裁决自动挂起。
这项标定的产出不是又一篇机制结论,而是一张可行性地图:它告诉后续所有三质 ΔAUC 实验,在当前标签质量下,哪些问题在多大样本量内根本不该立项。可裁决区的边界一旦画出,”三质判别力谁更强”的争论就不再靠立场说话,而靠预算说话。
L1 网络状态:L1 对应网络中无本主题(ΔAUC 最小可检出效应×样本量收缩曲线×金标准信度)对应边;仅相邻背景边 E-060 标签信度方法学(κ 型系数稳健性对样本量与类别数的敏感性、样本量单位错配)。该主题尚未进入 L1 对应网络。
证据与局限
已验证: 相关 ROC 的 DeLong 非参数比较框架与协方差估计成熟(PMID 3203132);双正态 ROC 样本量公式成熟(PMID 9249923);AUC 样本量预估对相关参数敏感、只能给界(PMID 20160839)。
部分验证: 不完美金标准导致 ROC/AUC 估计偏倚及其非参数估计框架(PMID 16011710);参照不完美时 AUC 差值符号可识别性(PMID 38994641);密度比模型似然校正(PMID 41351166)——方法学在诊断研究已落地,但均未在体质标签场景实测。
待验证: kappa=0.28 信度下可检出 ΔAUC 的具体收缩曲线与可裁决区边界,本文只给出模拟设计,尚无实测数据;标签翻转率由 kappa 反推的实现方式对结果的影响;配对与独立设计功效差在体质场景的幅度。
相关阅读
证据等级: C
参考文献: PMID 3203132, PMID 9249923, PMID 20160839, PMID 16011710, PMID 38994641, PMID 41351166
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Three-Quality ΔAUC Head-to-Head Minimum Detectable Effect and Sample Size Feasibility Boundary: Sample Size Shrinkage Curve for Detectable ΔAUC under Kappa=0.28 Gold Standard Re…
Evidence level: C
Minimum Detectable Effect and Sample Size Feasibility Boundaries for Head-to-Head ΔAUC Comparison of Three-Quality Pattern-Timing Frameworks (kappa = 0.28)
Under the traditional consensus that reliability is low (kappa = 0.28), whether a head-to-head comparison of ΔAUC between the three-Quality Pattern-Timing frameworks can yield a decisive verdict depends on the product of two factors: the dilution magnitude caused by label noise on ΔAUC, and the contraction rate of the standard error for DeLong’s test driven by sample size. This study uses simulation to plot the contraction curve of “detectable ΔAUC” against sample size, thereby pre-defining the regions where a decision is feasible versus those where it is not.
| Metric | Value / Description |
|---|---|
| Reliability (kappa) | 0.28 (traditional consensus: low agreement between raters) |
| Test statistic | ΔAUC (difference in Area Under the Curve) |
| Comparison type | Head-to-head (direct comparison of two models) |
| Feasibility boundary | Determined by label noise dilution × standard error contraction speed |
Evidence Level: B (Simulation-based inference; not yet validated by empirical field data)
Limitations: The contraction curves and decision boundaries derived here are theoretical projections based on simulated data under the assumption of kappa = 0.28. They do not account for real-world environmental variability or measurement artifacts that may further obscure ΔAUC differences.
Body
Head-to-head comparisons of the ΔAUC discriminative power, accounting for three quality criteria, presuppose a specific premise in all prior experimental designs (encoding schemes, stratification granularity, noise injection, imputation strategies): if a true effect exists, the experiment will detect it. This assumption holds broadly in domains with high gold-standard reliability but must be recalibrated for体质 labels (体质,traditional consensus framework for constitutional typing). Previous assessments report a gold-standard consistency of kappa≈0.28, indicating substantial misclassification between labels and true categories. Two consequences follow: first, label misclassification systematically depresses and distorts the observed AUC; this “imperfect gold-standard bias” is formally named and estimated in diagnostic methodology (PMID 16011710). Second, label noise dilutes between-group discriminative differences, shrinking the true ΔAUC on the observed scale. Recent methodological advances offer partial remedies: under imperfect reference standards, the sign of the difference between two AUCs remains identifiable and can support relative superiority testing (PMID 38994641); likelihood-based methods under density ratio models also enable corrected estimation of ROC/AUC (PMID 41351166). Yet no correction alters the harder constraint of statistical power.
The standard error of the ΔAUC test is determined by three quantities: the variances of the two individual AUCs and their covariance, which cannot be ignored in correlated ROC designs. The nonparametric framework for comparing correlated curves is the DeLong covariance method (PMID 3203132), while sample size estimation relies on the classic formula derived from bivariate normal ROC metrics (PMID 9249923). Crucially, the ΔAUC detection threshold contracts approximately as 1/√n, and sample size estimation is highly sensitive to the two correlation parameters; even slight deviations in assumptions can alter required sample sizes by several orders of magnitude, yielding only bounds rather than precise values (PMID 20160839). Overlaying this curve with the kappa≈0.28 label dilution yields the calibration target of this study: the same true ΔAUC shifts from “detectable” to “undecidable” across different sample size grids.
The simulation design comprises three layers. Layer one sweeps the true ΔAUC gradient across five levels: 0.02, 0.04, 0.06, 0.08, and 0.10, covering the span from “fine” to “coarse” effects. Layer two sweeps the gold-standard kappa gradient across four levels: 0.2, 0.4, 0.6, and 0.8; the 0.2 level represents current status (conservatively near 0.28), while the 0.8 level represents the upper bound of label quality. Layer three sweeps sample size across five levels per group: 100, 200, 400, 800, and 1600. The three layers combine into 100 grid points; each point undergoes 1,000 repeated simulations: first, true labels are flipped into observed labels according to the kappa level, then the power of the two-model AUC comparison and DeLong correlation test is computed on the observed labels. Two adjudication rules are pre-registered: a grid point with power ≥0.80 is marked “decidable region”; power <0.50 is marked “undecidable region,” where any result failing to detect difference must not be interpreted as negative evidence. The intermediate zone is a gray band requiring doubled sample size or pooled data before adjudication.
Three implementation details must be specified in the protocol. First, the label flip rate must be derived from kappa and explicitly implemented in simulations; merely discounting AUC is insufficient because misclassification compresses AUC nonlinearly and depends on disease prevalence. Second, the DeLong test must use a paired design on the same cohort; otherwise, the covariance term becomes invalid, and applying the paired formula to two independent samples underestimates variance and inflates power artificially. Third, when the reference standard is imperfect and a “superiority” conclusion is required, one must adopt either the proven sign-test of AUC differences or the likelihood correction framework (PMID 38994641; PMID 41351166), reporting both metrics side-by-side. If the “observed-label” and “corrected-label” conclusions contradict, adjudication is automatically suspended.
This calibration output is not another mechanistic conclusion but a feasibility map: it delineates, for all future three-quality ΔAUC experiments under current label quality, which questions fundamentally should not be launched at which sample sizes. Once the boundaries of the decidable region are drawn, debates over “which discriminative power is stronger” shift from positional arguments to budgetary constraints.
L1 Network Status: L1 corresponds to the absence of edges linking this topic (ΔAUC minimum detectable effect × sample size contraction curve × gold-standard reliability) within the L1 network; only adjacent background edge E-060, concerning label reliability methodology (κ-coefficient robustness sensitivity to sample size and category count, sample unit mismatch), has been incorporated. This topic has not yet entered the L1 network.
Evidence and Limitations
Verified: The DeLong nonparametric comparison framework for correlated ROC curves and covariance estimation is mature (PMID 3203132); the sample size formula for dual-normal ROC is established (PMID 9249923); however, AUC sample size estimation is sensitive to correlated parameters and can only provide bounds (PMID 20160839).
Partially Verified: Imperfect gold standards introduce bias in ROC/AUC estimation and their nonparametric frameworks (PMID 16011710); the sign of AUC differences remains identifiable under imperfect reference standards (PMID 38994641); likelihood correction via density ratio models is available (PMID 41351166). Methodology in diagnostic research is established, but not yet tested in the context of constitution labels.
Pending Verification: With a reliability coefficient of kappa=0.28, the specific shrinkage curve for detecting ΔAUC and the boundaries of the adjudicable region remain undefined; this paper provides only a simulated design, lacking empirical data. The impact of the implementation method for deriving label flip rates from kappa is also untested, as is the magnitude of power differences between paired and independent designs in constitution scenarios.
Related Reading
- AUC Advantage Zeroing Calibration: Noise Injection Crosspoint Arbitration
- Directional Encoding Tiered Granularity Optimal Scanning
- Interpolation Strategy as Independent Fairness Source Arbitration
- Cross-Cultural Frequency Cognition Research Pool
Evidence Level: C
References: PMID 3203132, PMID 9249923, PMID 20160839, PMID 16011710, PMID 38994641, PMID 41351166
Evidence Level: A = directly supported by PubMed, B = scientific analogy/indirect evidence, C = traditional consensus/preliminary exploration.
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.