折减口径的覆盖-宽度解耦检查:省下的宽度有没有偷走 95% 的信誉

Written by

in

Language

Evidence level: C

任何把区间「变窄」当成收益来申报的口径,都必须同时申报覆盖率——只报宽度的折减方案,等于允许用置信水平的透支来伪装成方法学进步。

正文

前一篇(设计效应标定误差的侵蚀)已经指出:当标定实验估出的组内相关系数 ρ̂ 被代入折减公式时,侵蚀分两档走——第一档是折减收益缩水,第二档更隐蔽,叫「覆盖与宽度解耦」:区间名义上变窄了,但它实际盖住真值的频率(覆盖率)可能已经跌破标称的 95%。这一篇专门裁决第二档:怎么检查折减口径有没有「偷走信誉」。

先说机制。折减口径的逻辑是:标定实验额外估出了 ρ̂,把这些信息代入设计效应公式(约等于 1 加上会话数减 1 乘 ρ),重复信息不再全额计入方差,区间因此变窄。但这里有个静默假设——把 ρ̂ 当成了真值。ρ̂ 本身有抽样误差:若真实 ρ 比 ρ̂ 大,设计效应被低估,方差该缩的没缩够,区间还是宽的,问题不大;可若真实 ρ 比 ρ̂ 小,折减把方差压得比实际允许的更窄,区间就会系统性过窄,覆盖率跌破 95%。糟糕的是,「省宽度」与「丢覆盖」在报表上长得很像:两者都表现为数字变小。只考核宽度的口径,会奖励那些把 ρ̂ 压到低位、区间好看但实际不可信的做法——这就是「偷工减料被奖励」的完整链条。

用一个具体数字看透支有多快。假设判读场景里每例被读 3 遍、真实组内相关是 0.3,设计效应约为 1.4,方差按此膨胀。若标定实验因会话数不足,把 ρ̂ 估成了 0.1,设计效应就掉到约 1.2——方差被压掉约 15%,区间等比变窄,报表上这是「漂亮的一轮改进」。但覆盖率的惩罚是非线性的:对正态近似区间,方差被低估 15% 大约意味着真实置信水平从 95% 掉到 91% 附近;低估 30% 就逼近 85%。也就是说,标定越省钱(会话数越少、ρ̂ 越不可信),折减口径的账面收益越大、真实信誉漏得越快,两者是同一枚硬币的两面。这就是为什么检查必须成对申报:单看宽度,你永远看不出硬币已经翻了面。

这不是我们发明的担忧,而是整类「把参数不确定性计入设计」问题的通用形态。群组随机试验文献里,当只有一个 ICC 估计可用时,用它的置信上限做保守样本量估计是可行但偏低效的路径(PMID 34477455);更新的 assurance / hybrid 方法干脆把参数不确定性直接纳入功效与样本量的计算,而不是事后假装参数已知(PMID 39935065)。信度研究领域同样如此:按预期置信区间宽度规划研究的框架(PMID 11746313)与精度导向的样本量设计(PMID 24338600)都强调,宽度目标只有建立在正确的方差结构上才有意义;而样本量不足时重测信度估计本身会失真(PMID 40119099)。共同教训是:参数有误差时,「名义窄」和「真覆盖」是两回事。

落到我们的判读场景,检查方案是仿真:沿用前一篇的 ρ̂ 抽样误差网格,在每一档标定会话数 n_s 上同时记录两个量——实际覆盖率(名义 95% 区间真正盖住真值的比例)和平均宽度。把两个量画在同一张图上,得到一条「覆盖-宽度前沿」:每一个口径策略对应前沿上的一个点,要么多覆盖、要么多省宽,不可能兼得。判据由此定死:若 n_s 达到 20 时覆盖率仍不低于 93%,且宽度还保留了已知 ρ 下理论收益的七成以上,折减口径可以在附加「覆盖检查」条款后放行;若覆盖率跌破 90%,说明折减在系统性透支置信水平,必须改用保守校准——例如按 ρ 估计的上限而非点估计来折算设计效应——才允许进入申报规范。

对读者的意义:当我们宣称「标定让结论更精准」时,这句话只有在覆盖率不缩水的前提下才成立。覆盖-宽度前沿就是防止「精准」这个词被滥用的审计工具——省下的宽度,必须是用真信息换的,不是用信誉换的。

证据与局限

已验证: 把 ICC 不确定性计入设计效应的必要性在群组随机试验领域有直接文献支撑(PMID 34477455);assurance 类方法处理参数不确定性的框架已成熟(PMID 39935065);信度研究的宽度导向样本量规划是既成方法学(PMID 11746313、24338600)。

部分验证: 「覆盖-宽度前沿」作为折减口径的放行判据是本系列提出的设计,其阈值(覆盖 93% 或 90%、收益保留 70%)为工作约定,未经独立文献校准;覆盖率对 ρ̂ 偏差方向的敏感性边界需仿真确定。

待验证: 折减口径在 ρ̂ 抽样误差网格下的实际覆盖率曲线;保守校准(按上限折算)相比点估计折算的宽度代价;n_s 台阶位置与覆盖安全线的联合取舍(承接下一篇最小样本量台阶定位)。

相关阅读

证据等级: C
参考文献: PMID 34477455, PMID 39935065, PMID 24338600, PMID 11746313, PMID 40119099

L1 网络状态:该主题尚未进入 L1 对应网络(CORRESPONDENCE_NETWORK.md 与 EDGE_TABLE_science-yixue.md 中无「折减口径 × 覆盖-宽度解耦」对应边);最近邻为 E-061「呼吸节律(0.1 Hz 谱峰)→ Mayer 波 / 自主神经谱」,状态 L2+ / open,原文摘录:「判别规格由『屏气 vs 定速呼吸』升级为『MSNA + HRV 分频段 + 呼吸操控 + 血压』四通道同测……判据尚未被联合实证检验,未升 L3」。本文属测量学方法课题,证据等级按规则封顶 C。

证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Decoupling Coverage-Width Check of Reduced Caliber: Did the Saved Width Steal 95% of Credibility?

Evidence level: C

Decoupling Coverage-Width in Reduced Aperture: Has the Saved Width Stolen 95% of Credibility?

Any aperture that declares narrowing the interval as a gain must simultaneously declare coverage; reporting only reduced width equates to permitting confidence-level overdraft to masquerade as methodological progress.

Body

The preceding article (Erosion by Design-Effect Calibration Error) established that when the intraclass correlation coefficient estimate $\hat{\rho}$ derived from calibration experiments is substituted into the design effect reduction formula, erosion proceeds in two tiers: the first tier involves a shrinkage of the reduction benefit, while the second tier is more insidious, termed “decoupling of coverage and width”: although the interval nominally narrows, its actual frequency of covering the true value (coverage) may fall below the nominal 95%. This article specifically adjudicates the second tier: how to verify whether the reduction criterion has “stolen credibility.”

First, the mechanism. The logic of the reduction criterion is as follows: calibration experiments additionally estimate $\hat{\rho}$; substituting this information into the design effect formula (approximately $1 + (n-1)\hat{\rho}$) means redundant information is no longer fully counted in the variance, and thus the interval narrows. However, there is a silent assumption here—treating $\hat{\rho}$ as the true value. Since $\hat{\rho}$ itself possesses sampling error: if the true $\rho$ is larger than $\hat{\rho}$, the design effect is underestimated, the variance is not shrunk sufficiently, and the interval remains wide; this is not problematic. Conversely, if the true $\rho$ is smaller than $\hat{\rho}$, the reduction compresses the variance narrower than actually permitted, causing the interval to become systematically too narrow, and coverage drops below 95%. The problem is that “saving width” and “losing coverage” look very similar on the report: both manifest as smaller numbers. A criterion that assesses only width rewards approaches that suppress $\hat{\rho}$ to low values, yielding an attractive interval but one that is actually untrustworthy—this constitutes the complete chain of “rewarding cost-cutting.”

To illustrate how rapidly this overdraw occurs, consider a specific numerical example. Suppose in the interpretation scenario each case is read 3 times, the true intraclass correlation is 0.3, and the design effect is approximately 1.4, with variance inflated accordingly. If the calibration experiment, due to insufficient session count, estimates $\hat{\rho}$ as 0.1, the design effect drops to approximately 1.2—the variance is compressed by about 15%, and the interval scales down proportionally; on the report, this appears as “a beautiful round of improvement.” However, the penalty on coverage is nonlinear: for normal-approximation intervals, underestimating variance by 15% roughly implies the true confidence level falls from 95% to near 91%; underestimating by 30% approaches 85%. In other words, the more cost-saving the calibration (fewer sessions, less trustworthy $\hat{\rho}$), the larger the apparent gain of the reduction criterion and the faster the true credibility leaks; these are two sides of the same coin. This is why verification must be reported in pairs: looking at width alone, you can never see that the coin has flipped.

This is not a concern we invented; it is the generic form of the broader class of problems involving “incorporating parameter uncertainty into design.” In the literature on cluster randomized trials, when only one ICC estimate is available, using its confidence upper bound for conservative sample size estimation is feasible but suboptimal (PMID 34477455); newer assurance/hybrid methods simply incorporate parameter uncertainty directly into power and sample size calculations, rather than pretending parameters are known post hoc (PMID 39935065). Similarly, in reliability research: frameworks planning studies by expected confidence interval width (PMID 11746313) and precision-oriented sample size designs (PMID 24338600) both emphasize that width targets only make sense if built on the correct variance structure; moreover, when sample size is insufficient, test-retest reliability estimates themselves become distorted (PMID 40119099). The common lesson is: when parameters have error, “nominal narrowness” and “true coverage” are two different things.

Applied to our interpretation scenario, the verification scheme is simulation-based: using the $\hat{\rho}$ sampling error grid from the preceding article, for each calibration session count $n_s$, we simultaneously record two quantities—actual coverage (the proportion of the nominal 95% interval that truly covers the true value) and average width. Plotting both quantities on the same graph yields a “coverage-width frontier”: each criterion strategy corresponds to a point on this frontier, either offering more coverage or saving more width; it is impossible to achieve both. The criterion is thus fixed: if $n_s$ reaches 20 and coverage remains at least 93%, while width retains more than 70% of the theoretical gain under known $\rho$, the reduction criterion may be approved with an added “coverage check” clause; if coverage drops below 90%, it indicates the reduction is systematically overdrawn on confidence level, and conservative calibration must be adopted—for example, using the upper bound of $\rho$ rather than the point estimate to compute the design effect—before entry into the reporting specification.

Significance for readers: when we claim “calibration makes conclusions more precise,” this statement holds only if coverage does not shrink. The coverage-width frontier serves as an audit tool to prevent the word “precise” from being abused: any saved width must be exchanged for true information, not for credibility.

Evidence and Limitations

Verified: The necessity of incorporating ICC uncertainty into design effects is directly supported by literature in the field of cluster randomized trials (PMID 34477455); assurance-type methods for handling parameter uncertainty have established frameworks (PMID 39935065); and reliability research’s width-oriented sample size planning is an established methodology (PMID 11746313, 24338600).

Partially Verified: The “coverage-width frontier” as the release criterion for reduced bandwidths is a design proposed in this series; its thresholds (coverage of 93% or 90%, benefit retention of 70%) are working conventions not yet calibrated by independent literature; the sensitivity boundary of coverage to the direction of ρ̂ bias must be determined via simulation.

Pending Verification: The actual coverage curve of reduced bandwidths under the ρ̂ sampling error grid; the width cost of conservative calibration (converted using upper bounds) versus point-estimate conversion; and the joint trade-off between n_s step positions and the coverage safety line (carrying forward to the next minimum sample size step localization).

Related Reading

Evidence Level: C
References: PMID 34477455, PMID 39935065, PMID 24338600, PMID 11746313, PMID 40119099

L1 Network Status: This topic has not yet entered the L1 correspondence network (no corresponding edge for “discounting aperture × coverage-width decoupling” in CORRESPONDENCE_NETWORK.md and EDGE_TABLE_science-yixue.md); the nearest neighbor is E-061 “Respiratory Rhythm (0.1 Hz spectral peak) → Mayer waves / autonomic spectrum”, status L2+ / open, with original excerpt: “Discrimination specifications upgraded from ‘breath-hold vs constant-rate breathing’ to four-channel simultaneous measurement of MSNA + HRV frequency bands + respiratory control + blood pressure… the criteria have not yet been jointly empirically validated, so L3 has not been reached.” This article belongs to the measurement methodology topic; per rules, its evidence level is capped at C.

Evidence Level: A = directly supported by PubMed; B = scientific analogy / indirect evidence; C = traditional consensus / preliminary exploration.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *