Evidence level: C
0.95 这个门限是在「点估计时代」定下来的,从未按「区间估计 + 指定功效」重算过——把「验收」和「证明」分开之后,门限-功效-样本量是一张三维表,每一格说的都是不同的话。
正文
任何一套「人工判读要过验收」的规则里,都藏着一个数字。在脉诊、舌诊这类一致性验收场景里,这个数字通常是 kappa ≥ 0.95。但很少有人追问:0.95 是谁、在什么前提下定的?
先分清两件事:验收和证明。「证明」是研究者想说明真值有多高,报个点估计加置信区间就够了;「验收」是门禁——m̂ 高于门限才放行,这时你关心的不是真值是多少,而是这套门禁的误放率和漏放率。混淆两者的代价是:一个在「描述水平」语境下合理的数字,搬到「放行门槛」语境下可能完全不设防。
三维表才是完整的账。门限 T、样本量 n、功效这三个量互相咬死:给定 n 和 T,真值恰好等于 T 时通过率恒等于显著性水平 α(3%–5%)——这意味着门禁在边界处形同虚设;要达到 80% 功效让真值过线,真值须到 0.996 以上,现实中不存在。反过来,若坚持 n=480 时让真值 0.95 有八成把握通过,门限须从 0.950 下调到约 0.921。这三个数构成一张 n × T × 功效的三维表,每一格都在回答不同的问题:这格的误放率是多少、漏放率是多少、要求的真值下限是多少、最危险的「真值比门限低 2 个百分点」档位穿透率是多少。
文献里的先例都是二维的,而且各说各话。早期的 kappa 样本量表在点估计框架下工作,先给出门限再配一个够用的 n;后来的路线改用置信区间精度定标——预先指定 κ 的置信区间上下限宽度来反推样本量,这正是现代精确样本量算法的思路;还有工作专门研究「训练到某个 kappa 准则」需要多少测试病例,发现准则定在 0.60 下界、要 95% 把握时,每位评定者至少要测 5 个病例以上,且病例在难度档位上的分布会显著改变所需样本量。这些文献没有一篇算过「门限 × 功效 × 样本量」的全格表——它们各自固定了两个轴、只扫一个轴。
为什么边界处通过率恒为 α?这反直觉但必然。把「验收」写成分位数的语言:精确下界方法给的是 m̂ 的置信区间下界,放行条件 = 下界 ≥ T。当真实一致性恰好等于 T 时,区间恰好有一侧概率质量骑在 T 上——在 α 的水平下,「下界超过真值」的概率由构造定义就是 α。也就是说:门禁只对「真值明显高于门限」的样本有拦截力,对「真值恰好在门限上」的样本近乎随机放行。这解释了为什么把惯例代入表后会出现「宣称而非验收」的尴尬:点估计时代只关心 m̂ 高不高,从不问「如果真值比门限低 2 个百分点,这套规则还拦得住吗」。穿底率这一格要的就是这个数——让「差一点点也放行」的漏洞可量化。
0.95 的门限演化也有考古价值。经典分期常把 0.81 以上称作 almost perfect,0.95 并不来自这张分期表;它更像是「再高一点更保险」的直觉放大,加上样本量规范缺位,就从习惯变成了行规。文献层面从未有人论证过「为什么是这个数、配多大 n、保证什么功效」——本主题的三维表正是替这个缺位补账。
把三条现行惯例代入这套表打分:「n=120 例 + 点估计 ≥0.95」「n=240 + 精确下界」「n=480 + 精确下界」。判据有三条:若任一惯例在「真值比门限低 2pp」处的通过率超过 20%,它就不构成验收、只构成宣称,须废止;若「n=240 + 精确下界」这一格在 80% 功效处所需真值已超 0.99,则本领域根本不该用 0.95 门限配精确区间,须改为「门限 0.90 + 精确区间 + n≥480」(宣称达标用)或「门限 0.95 + 点估计 + 强制报 CI」(描述水平用)两条路线择一;若功效 0.50 与 0.80 两档在 n≥480 处所需真值相差不足 1pp,功效这一轴可压掉,三维表退化成二维。
这不是本领域特有的麻烦:序贯监测的模拟研究已明确报告,计划块数少时 I 类错误可显著膨胀、块数多或加校正后才能回到名义值——门限与设计参数的耦合是普遍现象。0.95 没什么神圣的,它只是从点估计时代 inherited 下来的一个习惯数;要么重算它,要么写清楚它在哪个功效下守护什么。
证据与局限
已验证:kappa 置信区间宽度随样本量变化可精确计算(PMID 24338600、22560852、1410963);「训练到 kappa 准则」的样本量-病例数关系已有模拟结果(PMID 20473138)。
部分验证:门限边界处通过率恒为 α、功效-真值需求曲线的解析式成立,但本领域 n ∈ {60…1920} 全格表尚未有人算过。
待验证:三条现行惯例的穿透率打分;「宣称/描述」双路线的适用边界;功效轴是否可压掉。
L1 网络状态:该主题尚未进入 L1 对应网络(方法学/统计定标类主题,非易学-科学对应边)。L1 中最接近的边为 E-060 标签前提(conflict):「样本量单位错配(n≥50 指评定者 vs 120 指病例)与精度区间错配(『一致性 low』正落在加权系数精度最差区间)」——与本篇「门限数字从未按功效重算」同属验收规格错配家族,但不是同一条边,已登记缺口表。
相关阅读
证据等级: C
参考文献: PMID 15733050, PMID 24338600, PMID 22560852, PMID 20473138, PMID 1410963
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Threshold 0.95: Acceptance, Proof, and Efficacy – A Three-Dimensional Table of Sample Size
Evidence level: C
The threshold of 0.95 was established during the era of point estimation and has never been recalculated using interval estimation plus specified power. Once acceptance and proof are separated, the threshold-power-sample size relationship becomes a three-dimensional table, where each cell conveys different information.
Body
Any set of rules requiring “artificial judgment to pass acceptance” contains a numerical threshold. In consistency-based acceptance scenarios such as pulse diagnosis or tongue diagnosis, this number is typically kappa ≥ 0.95. Yet few ask: who set 0.95, and under what conditions?
First, distinguish two concepts: acceptance and proof. “Proof” refers to the researcher’s attempt to demonstrate how high the true value is; reporting a point estimate with a confidence interval suffices. “Acceptance” acts as a gate—m̂ must exceed the threshold to be allowed through. Here, concern is not with the true value itself, but with the false acceptance rate and false rejection rate of this gate. Confusing the two carries a cost: a number reasonable in a “descriptive level” context may become completely unguarded when shifted to a “release threshold” context.
The three-dimensional table constitutes the complete account. Threshold T, sample size n, and power are interlocked: given n and T, if the true value exactly equals T, the pass rate is always equal to the significance level α (3%–5%)—meaning the gate is virtually non-existent at the boundary. To achieve 80% power for the true value to pass, the true value must exceed 0.996, which does not exist in reality. Conversely, if one insists on n=480 to give the true value 0.95 an 80% chance of passing, the threshold must be lowered from 0.950 to approximately 0.921. These three quantities form an n × T × power table, where each cell answers different questions: what is the false acceptance rate? What is the false rejection rate? What is the required lower bound of the true value? What is the penetration rate for the most dangerous “true value 2 percentage points below the threshold”?
Precedents in literature are all two-dimensional, and each speaks its own language. Early kappa sample size scales operated under the point estimation framework, first setting the threshold then matching it with a sufficient n. Later approaches switched to calibrating via confidence interval precision—pre-specifying the width of κ’s confidence interval limits to back-calculate sample size, which is precisely the logic of modern exact sample size algorithms. Other work specifically studied how many test cases are needed to “train to a kappa criterion,” finding that with a lower bound criterion of 0.60 and 95% confidence, each rater must assess at least five cases, and the distribution of cases across difficulty levels significantly alters the required sample size. None of these papers calculated the full table of “threshold × power × sample size”—they each fixed two axes and scanned only one.
Why is the pass rate at the boundary always α? This is counterintuitive but inevitable. Writing “acceptance” in terms of quantiles: exact lower bound methods provide the lower confidence limit for m̂, and the release condition is lower bound ≥ T. When the true consistency exactly equals T, the interval has precisely one side’s probability mass riding on T—at level α, the probability that “the lower bound exceeds the true value” is defined by construction as α. In other words: the gate only intercepts samples where the true value is clearly above the threshold; for samples where the true value lies exactly on the threshold, it lets them through almost randomly. This explains why substituting conventions into the table leads to the awkwardness of “claiming rather than accepting”: the point estimation era cared only whether m̂ was high, never asking “if the true value is 2 percentage points below the threshold, can this rule still catch it?” The “penetration rate” cell specifically requires this number—to quantify the loophole where “almost enough also passes.”
The evolution of the 0.95 threshold also has archaeological value. Classic classifications often label anything above 0.81 as “almost perfect,” yet 0.95 does not originate from this classification table; it seems more like an intuitive amplification of “a bit higher is safer,” compounded by the lack of sample size standards, turning habit into convention. No literature has ever argued “why this number, matched with what n, may support what power”—this subject’s three-dimensional table precisely fills that gap.
Substituting the three current conventions into this table for scoring: “n=120 cases + point estimate ≥0.95”, “n=240 + exact lower bound”, “n=480 + exact lower bound”. Three criteria apply: if any convention’s pass rate at “true value 2pp below threshold” exceeds 20%, it does not constitute acceptance but merely claims, and must be abolished; if the “n=240 + exact lower bound” cell requires a true value exceeding 0.99 at 80% power, this field should not use the 0.95 threshold with exact intervals at all, but instead choose between “threshold 0.90 + exact interval + n≥480” (for claims meeting standards) or “threshold 0.95 + point estimate + mandatory CI reporting” (for descriptive level use); if the difference in required true values between power levels 0.50 and 0.80 at n≥480 is less than 1pp, the power axis can be collapsed, reducing the three-dimensional table to two dimensions.
This is not a trouble unique to this field: sequential monitoring simulation studies have clearly reported that with few planned blocks, Type I error can inflate significantly, and only with many blocks or correction does it return to the nominal value—the coupling of threshold and design parameters is universal. 0.95 is not sacred; it is merely a habitual number inherited from the point estimation era: either recalculate it, or clearly state under what power level it guards what.
Evidence and Limitations
Verified: kappa confidence interval width varies with sample size and can be precisely calculated (PMID 24338600, PMID 22560852, PMID 1410963); the sample size-case number relationship for “training to kappa criteria” has simulation results (PMID 20473138).
Partially verified: the pass rate at threshold boundaries is always α, and the analytical expressions for power-true value demand curves hold, but no one has yet calculated the full table for n ∈ {60…1920} in this field.
Pending verification: penetration rate scoring for the three current conventions; the applicability boundaries of the “claim/describe” dual routes; whether the power axis can be collapsed.
L1 Network Status: This topic has not yet entered the L1 corresponding network (methodological/statistical calibration topics, not Yijing-science correspondence edges). The closest edge in L1 is the E-060 label premise (conflict): “sample unit mismatch (n≥50 refers to raters vs 120 refers to cases) and precision interval mismatch (‘consistency low’ falls precisely in the worst precision zone of weighted coefficient intervals)”—this belongs to the same family of acceptance specification mismatches as “threshold numbers never recalculated based on power,” but it is not the same edge; the gap has been registered in the deficit table.
Related Reading
- GDV Residual Threshold and Sample Size Calibration
- Operationalization and Pre-registration of Divination Accuracy Thresholds
- Research Pool ①: Resonance Consciousness Field
Evidence Level: C
References: PMID 15733050, PMID 24338600, PMID 22560852, PMID 20473138, PMID 1410963
Evidence Level: A=direct PubMed support, B=scientific analogy/indirect evidence, C=traditional consensus/preliminary exploration
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply