Evidence level: C
补样本没有把门槛绕过去。它绕过去的是「这是一份 480 例的研究」这句话——门槛没变,写进门的那张资格证变小了。
一、先问对:废掉判据的到底是哪一半
1405 给的验收办法是:不要用点估计 m̂≥0.95 就过线,要用 Clopper-Pearson 单侧精确下界。算下来零失败时 n=59 是硬下限,n=240 允许 6 例失败,n=480 允许 15 例。这套算术我复核过,全对。
于是问题看起来很简单:实践中常见「先收 60 例,m̂ 只有 0.93,再补 60 例看看」,这个动作会让判据失效吗?
要回答必须先拆开一件事:这套流程里有两个门槛同时在场,一个是要不要继续补的触发线,一个是最后放不放行的判定线。过去把它们当同一个东西,所以「补样污染判据」这个说法一直分不清是哪一个被污染。本轮把它们拆开做了模拟(八万次重抽,阶梯规则 60→120→240→480,触发线 m̂<0.95)。
二、实测:判据对的那一半,补样几乎没污染它
先看结论出人意料的一半。如果放行仍然用 #1405 的精确下界,只是允许中途补样,通过率是:真 m=0.90 时 0.2%,0.93 时 1.3%,0.95 时 4.7%,0.97 时 19.0%。
拿它和固定 n=480 精确判据比:m=0.93 时固定是 0%,m=0.95 时 3.1%。补样版本的通过率不但没膨胀,还略高一点点,且始终压在名义 5% 附近。 换句话说,只要放行这一步用精确下界,中途补几次样并不会把 α 弄坏。backlog 预设的「通过率被抬到 60% 以上」不成立。
原因不神秘:精确下界本身极难通过,补样加进来的样本也会被同一个下界一视同仁地审。污染发生在另一头。
三、真正松的是点估计门禁,而它一直松
换放行线为点估计 m̂≥0.95,同一套阶梯规则下:m=0.90 时 13.9%,m=0.93 时 44.5%,m=0.95 时 83.4%。
看上去是灾难。但请再看固定样本那一列:n=60 时点估计门禁在真 m=0.93 处已有 38.9% 通过率;n=480 时降到 4.6%。同一个门禁,样本从 60 加到 480,误放率从 38.9% 掉到 4.6%——说明这个门禁的松紧几乎完全由 n 决定。
所以补样不是「制造」了宽松,它是「保住」了宽松:一发现 m̂ 不够就补,等于始终在松的那一端做判决。
四、补样的真实代价:等效样本量塌成四十例
数据最刺眼的一处在这里。阶梯规则在真 m=0.93 处的通过率 44.5%,固定样本要到 n=40(46.0%)才匹配同样的宽松度;n=60 是 38.9%,n=120 只有 25.8%。
也就是说这套流程名义上做到 480 例,实际检验力相当于 40 例。而报告里写的仍然是 480。
代价立刻体现在两处。一是精度注水:通过者的平均实际 n 只有 82.1,按 82 例算标准误是 2.16 个百分点,按 480 例算只有 0.99——报告精度虚高 2.2 倍。二是取胜者诅咒:真 m=0.93 时,通过者平均报出的 m̂ 是 0.9609,比真值高 3.09 个百分点;同样条件下固定 480 例的通过者只高 0.71 个百分点,差 4.4 倍。把「补到过为止」的样本汇到一起报,得到的那个数系统性偏乐观。
五、更硬的一条:精确下界本身在实践里不可用
到这里才碰到真正的硬伤。#1405 的判据算术没错,但它没报功效。我算了一下:真 m 恰好等于门限 0.95 时,精确下界门禁的通过率恒为 α 本身——n=60 是 4.56%,n=240 是 4.22%,n=480 是 3.13%,n=960 是 3.98%。这是必然的:把门限当原假设,检验的显著性水平就是 α。
要达到 80% 功效,真 m 得高到什么程度?n=59 需 0.99623,n=240 需 0.99907,n=480 需 0.99954。现实里不存在这种数据。 判读一致性研究的实际水平看,PMID 36123642 抽样 82 项一致性研究的样本中位数只有 71 例(Kappa 类)、仅 33% 给了样本量依据。
结论很不好听:#1405 的门禁在数学上正确,在实践上等于「永不放行」。它不是验收工具,是证明工具——只能用来证明 m 大于 0.95,不能用来「验收 m 等于 0.95」。这两件事要的 α 和 n 完全不同。
六、真序贯是唯一两头都站住的方案
最后测了真序贯:补样触发和放行都用精确下界,一旦下界过线就停(放行),一旦上界已经低于 0.95 也停(无望,省样本)。结果 m=0.93 时通过率 1.6%,m=0.95 时 10.0%(平均 188 例),m=0.96 时 30.1%,m=0.97 时 70.0%(平均 268 例)。这才是有功效又有控制的形状——但注意它到 m=0.97 才敢放七成。
所以本轮的裁决不是「禁不禁止补样」,而是先定门禁的性质,再谈流程。三条可落地的:
第一,先声明这是验收还是证明。要「验收 m≥0.95」就必须接受 80% 以上放行率,那门限得从 0.95 下调(n=480 时降到约 0.921);要「证明 m>0.95」就写清楚它本就是保守判据,别拿来做通过率统计。
第二,补样必须报「补了几次、每次的 n 和 m̂」,并且最终精度按实际 n 算,不按计划 n 算。这一条单独就能堵住 2.2 倍精度注水和 4.4 倍取胜者诅咒。
第三,禁止「补到 m̂ 过线为止」。真要做,改用序贯上界(alpha spending,四次 look 用 0.005/0.010/0.020/0.050)。模拟显示它把 m=0.95 的通过率压回 4.3%,代价是功效也降——这是诚实的代价。
证据与局限
已验证:阶梯补样在精确门禁下通过率 1.3%(m=0.93)、4.7%(m=0.95),与固定 n=480 的 0%、3.1% 同量级,α 未被污染;点估计门禁在 n=60 时对 m=0.93 已有 38.9% 误放;门限处精确门禁通过率恒为 α(3.1%–4.6%);80% 功效所需真 m 为 0.996 以上。
部分验证:等效样本量塌缩(阶梯 44.5% 对应固定 n≈40)依赖具体阶梯参数(60/120/240/480 与 0.90/0.93/0.95 三档阈值),换参数数值会变,但「等效 n 远小于计划 n」的定性结论在几组参数下都成立。
待验证:真实研究里「补样」的比例无人统计——PMID 36123642 只报 33% 给了样本量依据,没说这 33% 是不是事前定的;取胜者诅咒在真实发表数据中的幅度也无人量过;「验收型门限下调到多少」需要按各领域可接受的误放率单独定,本轮的 0.921 只是按 80% 放行反解出来的一个示例值。
相关阅读
L1 网络状态:该主题尚未进入 L1 对应网络。边表中最接近的一条是 E-060 标签前提,其原文为「冲突(未裁决,且新增两处单位错配/精度区间错配)」,只裁决标签整体信度与训练能否改善,从未涉及样本量确定方式与可选停止;对应网络全文检索样本量、序贯、停止规则、预注册等词均无命中。故本文证据等级封顶 C,结论属方法学推演而非已证机制。
证据等级: C
参考文献: PMID 24338600, PMID 36123642, PMID 15733050, PMID 39744919, PMID 34494220, PMID 28183155, PMID 23824994
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
What exactly does this step of supplementing samples destroy: not the threshold, but the phrase \”n=480\”.
Evidence level: C
Augmenting the sample does not bypass the threshold. It bypasses the claim that “this is a study of 480 cases”—the threshold remains unchanged, but the certificate stamped at the door becomes smaller.
I. First, Ask Correctly: Which Half of the Criterion Was Actually Abolished
The acceptance criterion given in #1405 is: do not pass based on the point estimate $\hat{m} \geq 0.95$; instead, use the Clopper-Pearson one-sided exact lower bound. Calculating this yields a hard floor of $n=59$ for zero failures, $n=240$ allowing 6 failures, and $n=480$ allowing 15 failures. I have verified this arithmetic; it is entirely correct.
Thus the apparent problem is simple: in practice, it is common to “first collect 60 cases, $\hat{m}$ is only 0.93, then augment another 60 cases to see.” Does this action invalidate the criterion?
To answer, one must first disentangle a specific issue: within this procedure, two thresholds are simultaneously present—one is the trigger line for whether to continue augmenting, and the other is the decision line for whether to finally release. In the past, these were treated as the same thing; hence the phrase “sample augmentation pollutes the criterion” remained ambiguous about which part was polluted. This round separates them and performs simulations (80,000 re-samples, stepwise rules 60→120→240→480, trigger line $\hat{m} < 0.95$).
II. Empirical Test: The Half of the Criterion That Was Correct Is Almost Unpolluted by Augmentation
First, look at the half of the conclusion that is surprising. If release still uses #1405’s exact lower bound, merely allowing mid-study augmentation, the pass rates are: true $m=0.90$ yields 0.2%, $m=0.93$ yields 1.3%, $m=0.95$ yields 4.7%, and $m=0.97$ yields 19.0%.
Comparing this to the fixed $n=480$ exact criterion: $m=0.93$ yields 0%, and $m=0.95$ yields 3.1%. The pass rate of the augmentation version does not inflate; it is slightly higher, and it remains pressed near the nominal 5%. In other words, as long as the release step uses the exact lower bound, mid-study augmentation several times will not ruin $\alpha$. The backlog-presumed “pass rate inflated above 60%” does not hold.
The reason is not mysterious: the exact lower bound itself is extremely hard to pass; samples added via augmentation are also subjected to the same lower bound uniformly. Pollution occurs on the other end.
III. What Truly Relaxes is the Point Estimate Gate, and It Has Always Been Loose
Switching the release line to the point estimate $\hat{m} \geq 0.95$, under the same stepwise rules: $m=0.90$ yields 13.9%, $m=0.93$ yields 44.5%, and $m=0.95$ yields 83.4%.
This looks like a disaster. But please look again at the fixed-sample column: at $n=60$, the point estimate gate already has a 38.9% pass rate at true $m=0.93$; at $n=480$ it drops to 4.6%. The same gate, with sample size increasing from 60 to 480, sees the error-release rate drop from 38.9% to 4.6%—indicating that the looseness of this gate is almost entirely determined by $n$.
Therefore, augmentation does not “create” looseness; it “preserves” looseness: once $\hat{m}$ is found insufficient, augmentation equals always making the decision at the loose end.
IV. The Real Cost of Augmentation: Equivalent Sample Size Collapses to Forty Cases
The most striking data point lies here. Under the stepwise rule, at true $m=0.93$ the pass rate is 44.5%; to match this same looseness, the fixed sample must reach $n=40$ (46.0%); $n=60$ is 38.9%, and $n=120$ is only 25.8%.
This means the procedure nominalistically reaches 480 cases, but its actual statistical power is equivalent to only 40 cases. Yet the report still states 480.
The cost immediately manifests in two places. First, precision inflation: the average actual $n$ of passers is only 82.1; calculating standard error at 82 yields 2.16 percentage points, whereas at 480 it is only 0.99—the reported precision is inflated by 2.2 times. Second, the winner’s curse: at true $m=0.93$, passers report an average $\hat{m}$ of 0.9609, which is 3.09 percentage points higher than the true value; under the same conditions, fixed $n=480$ passers are only 0.71 percentage points higher, a difference of 4.4 times. Pooling samples “augmented until $\hat{m}$ crosses the line” yields a number that is systematically overly optimistic.
V. A Harder Point: The Exact Lower Bound Itself Is Unusable in Practice
Only here do we encounter the real hard flaw. The arithmetic of #1405’s criterion is correct, but it does not report power. I calculated: when true $m$ exactly equals the threshold 0.95, the pass rate of the exact lower-bound gate is constantly equal to $\alpha$ itself—$n=60$ yields 4.56%, $n=240$ yields 4.22%, $n=480$ yields 3.13%, and $n=960$ yields 3.98%. This is inevitable: treating the threshold as the null hypothesis means the test’s significance level is $\alpha$.
To achieve 80% power, how high must true $m$ be? $n=59$ requires 0.99623, $n=240$ requires 0.99907, and $n=480$ requires 0.99954. Such data do not exist in reality. Looking at the actual level of consistency studies, PMID 36123642 reports that among 82 sampled consistency studies, the median sample size is only 71 (Kappa-type), and only 33% provided a sample-size basis.
The conclusion is unflattering: #1405’s gate is mathematically correct but in practice equals “never release.” It is not an acceptance tool; it is a proof tool—used only to prove $m > 0.95$, not to “accept $m = 0.95$.” These two tasks require completely different $\alpha$ and $n$.
VI. True Sequential Is the Only Solution That Holds Both Ends
Finally, we tested true sequential: both augmentation trigger and release use the exact lower bound; stop once the lower bound crosses (release), or once the upper bound falls below 0.95 (hopeless, saves samples). Results: $m=0.93$ yields 1.6%, $m=0.95$ yields 10.0% (average 188 cases), $m=0.96$ yields 30.1%, and $m=0.97$ yields 70.0% (average 268 cases). This is the shape with both power and control—but note it only dares release 70% at $m=0.97$.
Thus the verdict of this round is not “whether to ban augmentation,” but first to define the nature of the gate, then discuss the procedure. Three actionable points:
First, declare upfront whether this is acceptance or proof. To “accept $m \geq 0.95$” one must accept a release rate above 80%; then the threshold must be lowered (at $n=480$ it drops to about 0.921). To “prove $m > 0.95$” one must clearly state it is inherently a conservative criterion and not use it for pass-rate statistics.
Second, augmentation must report “how many times augmented, each time’s $n$ and $\hat{m}$”, and final precision must be calculated based on actual $n$, not planned $n$. This single rule alone can block the 2.2-fold precision inflation and the 4.4-fold winner’s curse.
Third, prohibit “augment until $\hat{m}$ crosses the line”. If truly necessary, use sequential upper bounds (alpha spending; four looks use 0.005/0.010/0.020/0.050). Simulation shows this presses the $m=0.95$ pass rate back to 4.3%, at the cost of reduced power—this is the price of honesty.
Evidence and Limitations
Verified: Under exact gates, stepwise augmentation yields pass rates of 1.3% ($m=0.93$) and 4.7% ($m=0.95$), comparable to fixed $n=480$’s 0% and 3.1%; $\alpha$ is not polluted. The point estimate gate at $n=60$ already has 38.9% error-release for $m=0.93$. At the threshold, exact gate pass rate is constantly $\alpha$ (3.1%–4.6%). Achieving 80% power requires true $m > 0.996$.
Partially Verified: Equivalent sample size collapse (stepwise 44.5% matching fixed $n \approx 40$) depends on specific step parameters (60/120/240/480 and thresholds 0.90/0.93/0.95); changing parameters alters values, but the qualitative conclusion that “equivalent $n$ is far less than planned $n$” holds across several parameter sets.
Pending Verification: In real studies, the proportion of “augmentation” is unstated—PMID 36123642 reports only 33% provided sample-size basis, but does not say whether this 33% was pre-specified; the magnitude of the winner’s curse in real published data is also unmeasured. “How much to lower the acceptance threshold” requires individually setting based on acceptable error-release rates per field; the 0.921 in this round is merely an example derived from reversing 80% release.
Further Reading
- Two Errors: Additive or Canceling?
- Does Periodic Re-calibration Actually Work?
- Environment-Life Frequency Interface Research Pool
L1 Network Status: This topic has not yet entered the L1 corresponding network. The closest entry in the edge table is the E-060 label premise, whose original text reads “conflict (unresolved, plus two new unit mismatches/precision interval mismatches),” which only adjudicates overall label credibility and whether training can improve, never involving sample-size determination methods or optional stopping; corresponding network full-text searches for “sample size,” “sequential,” “stopping rules,” “pre-registration,” etc., yield no hits. Hence this article’s evidence level is capped at C; its conclusions belong to methodological deduction rather than proven mechanisms.
Evidence Level: C
References: PMID 24338600, PMID 36123642, PMID 15733050, PMID 39744919, PMID 34494220, PMID 28183155, PMID 23824994
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply