tau 这个数测得出来,那条线判不过去:门槛附近九成五落在灰区,而判「过严」天生比判「成立」难

Written by

in

Language

Evidence level: C

前面几篇把「以资深者为准能错多少」算到了小数点后一位,可那个数从头到尾建立在一个从没被测过的参数上:跨场景异质性 tau。这篇不谈它该多大,只回答一个更前置的问题——给它测一个值,这件事本身可不可做,测出来又能判到什么程度。

一、那条线不是竖线,是斜线

先把上一稿里的临界值摆正。错误率等于两倍的 p 乘一减 p,其中 p 是标准正态分布里 mu 除以 tau 那个位置的概率。令它等于百分之十五,解出来不是「tau 小于等于三点五九个百分点」,而是mu 除以 tau 大于等于一点三九四。

这个换算一写出来,原来那句话的问题就露了:三点五九这个数字,只在 mu 恰好等于五个百分点时成立。mu 换成两个百分点,门槛掉到一点四三;换成十个百分点,门槛抬到七点一七。同一句「tau 小于三点五九故禁令过严」,在 mu 等于八个百分点时对应的真实错误率只有百分之二点六,在 mu 等于两个百分点时却是百分之四十一。

所以回溯不能只交一个 tau。每一组数据必须带上它自己的 mu,否则那个绝对门槛是借来的量纲——和上一篇在 logit 换算上踩的是同一个坑。

二、三十组够测 tau 吗:够,中位数几乎不偏

假设每组三十例、单次判读噪声八个百分点,用经典矩估计跑三千次模拟。真 tau 为二时,点估计的中位数是一点九四;为三时二点九四;为五时四点九二。偏差小到可以忽略。

区间估计也在名义水平上:三种真值下覆盖率都在百分之九十四点八到九十五点一之间。方法学上有明确背书——九种异质性方差估计量的系统比较推荐限制最大似然,并明确指出矩估计在小样本、稀有结局时系统性偏低(PMID 30067315,Langan 等,Res Synth Methods 2019);十六种方法的比较则直接建议把区间和点估计一起报(PMID 26332144,Veroniki 等)。Q 剖面法做区间本身是标准做法,但 Q 统计量一族的方法在实践条件下覆盖率会明显低于名义值(PMID 30589219,van Aert 等),所以这里的百分之九十五是乐观上限。

结论:测 tau 不难。三十组、每组三十例,够用了。

三、难的是「判过没过线」,而良性和有害不对称

这才是本篇真正的发现。

对着三点五九这条线做判定,如果要求区间整体落在门槛一侧才算判出,结果是这样的:真 tau 等于二(禁令其实过严,错误率仅百分之一点二),三十组时判对只有百分之七十三点八,四分之一被判成「禁令成立」。真 tau 等于五(禁令确实成立,错误率百分之二十六点七),同样三十组,判对百分之九十六点一,误判只剩百分之三点九。

一边七十四,一边九十六。原因很朴素:判「过严」要用区间的上界去低于门槛,而上界天生就宽——三十组时上界的中位数是点估计的一点四四倍;判「成立」只需要点估计越过门槛。

也就是说,这套流程对「维持禁令」宽松,对「推翻禁令」严苛。举证责任挂在想推翻的那一方身上,而且是无意识挂上去的。 这跟统计里常见的保守方向相反:通常保守是「不敢说有效」,这里是「不敢说没问题」。

更要紧的是,门槛正上方那段几乎测不出来。真 tau 恰好等于三点五九时,百分之九十四点九的样本落在灰区——既不能说它过严,也不能说它成立。而这个区间恰恰是唯一真正需要答案的地方。

四、加病例几乎没用,瓶颈是组数

直觉会认为多收病例就能把 tau 测准。算一下就知道不行。

固定三十组,每组例数从三十加到一千九百二十(六十四倍),mu 的估计标准误只从零点七六降到零点六六个百分点。原因是 tau 平方在总方差里的份额从百分之七十五涨到百分之九十九点五——噪声项被压干了,剩下的全是真异质性,而它只能靠增加组数来平均掉。

要把「错误率区间宽度」压到十个百份点以内,每组三十例时需要八十组;每组六十例时需要五十组。要判出「良性」达到九成五把握,真 tau 等于二需要八十组,等于三需要五百组,等于三点四需要八百组以上。

越接近门槛,越判不出。 这不是精度问题,是这套判据的结构问题。

五、小样本还会把良性伪装成完美

还有一层。经典矩估计在小 K 下会被零截断——真 tau 等于一、K 等于十时,四成样本的 tau 被压成零。零意味着「各组完全一致、没有任何跨场景波动」,读起来是最理想的状态。

于是出现一种反向失真:真正最良性、波动最小的那批数据,有相当比例被报成「波动为零」。而零这个值在业务上会直接被读成「这条禁令纯属多余」。

六、能给的与不能给的

能给的:一个三十到八十组、每组三十例的回溯,可以把 tau 从假想参数变成带区间的实测值,中位数基本无偏。这是可做的。

不能给的:用同一个回溯去裁决「禁令该留该废」。门槛附近判不出,良性一侧四分之一错判,而加病例解决不了。

正确用法:不问「tau 有没有超过三点五九」,改问「mu 除以 tau 的区间在哪里」,并且报告这个比值而不是 tau 的绝对值。比值在八十组时区间仍是一点零六到一点八六,对应错误率百分之二十五到百分之六——即便如此宽,它至少不借量纲、不随 mu 漂移。

证据与局限

已验证: 矩估计在中小 K 下点估计近似无偏、Q 剖面区间覆盖率接近名义值,这两点在本篇模拟与方法学文献(PMID 30067315、PMID 26332144)中一致;限制最大似然优于矩估计是共识结论。

部分验证: 「判良性比判有害难」这一不对称在本篇模拟中稳定复现(三十组时七十四对九十六),但它依赖「区间整体落在一侧才算判出」这条判定规则。换用其他规则(例如只看点估计)不对称会消失,代价是失去控制。判定规则本身没有外部标准可依,属本篇设定。

待验证: ① mu 与 tau 在真实判读数据里是否独立——若 mu 与 tau 相关(严厉的中心波动也大),比值判据的分布会变形,本篇按独立生成,未验;② 三十组的历史数据在文献里是否找得到——多数一致性研究报告的是合并 kappa 而非分组原始差,回溯可能根本凑不齐 K;③ 本模拟假设每组等例数、等噪声,真实数据里例数与噪声同时变化,加权方式需另行检验。

相关阅读

证据等级: C
参考文献: PMID 30067315, PMID 26332144, PMID 30589219, PMID 28122727
L1 网络状态:CORRESPONDENCE_NETWORK.md(1956 行)逐词统计:「异质」0 处、「随机效应」0 处、「tau」0 处、「错误率」0 处、「kappa」0 处、「方差」1 处(第 1694 行 E-005 声音韵律条目,指基频方差这一声学量,与跨场景方差成分无关)、「偏倚」1 处(第 265 行 KB-338 自陈叙述性综述无偏倚评估)、「评定者」4 处、「评分者」1 处、「判读」1 处(第 1832 行 E-061 驱动方向判读须跨两族方法),全部落在 RSA 读数规格与穴位对照臂清单层。EDGE_TABLE_science-yixue.md(63 行)grep「kappa/信度/一致/方差/误差/偏倚/校正/异质/随机效应」全部 0 命中。该主题最接近的边仍是 E-060 标签前提(五脏辨证作分组标签|前提|标签信度,状态 conflict 未裁决)——KB-317 记两处错配、KB-333 记序数分类下系数选择与估计精度须分层裁决、KB-334 要求 AC2 与患病率分报;该边只登记「标签信度够不够用」,未登记「跨场景异质性有多大、以及它能否被实测出来」。该主题尚未进入 L1 对应网络,已登记至 science-yixue-l1-gaps.md(「测量一致性」簇第 60 条)。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Tau is measurable, but that line cannot be crossed: ninety-five percent of the threshold falls in the gray zone, and judging \”too strict\” is inherently harder than judging \”esta…

Evidence level: C

Previous articles calculated “how much error is acceptable for experts” to one decimal place. However, that number relies entirely on an untested parameter: cross-scenario heterogeneity tau. This paper does not determine its magnitude; it answers a more fundamental question: Can we measure tau at all, and if so, what can the result tell us?

1. The line is not vertical; it is slanted

First, correct the threshold from the previous draft. The error rate equals twice p times (1 − p), where p is the probability at the position of mu divided by tau in the standard normal distribution. Setting this equal to 15% yields not “tau ≤ 3.59%” but mu / tau ≥ 1.394.

This conversion exposes the flaw in the previous statement: the figure 3.59 holds only when mu equals exactly 5%. If mu is reduced to 2%, the threshold drops to 1.43; if mu increases to 10%, the threshold rises to 7.17. The claim “tau ≤ 3.59, so the ban is overly strict” is therefore misleading. When mu = 8%, the true error rate corresponding to this statement is only 2.6%; when mu = 2%, it jumps to 41%.

Thus, one cannot simply report tau. Each dataset must include its own mu, otherwise the absolute threshold borrows units from another context—falling into the same pit as the logit conversion in the previous paper.

2. Are thirty groups enough to estimate tau? Yes, the median is nearly unbiased

Assuming 30 cases per group and single-reading noise of 8%, running 3,000 simulations via classical moment estimation yields:
– True tau = 2 → median point estimate ≈ 1.94
– True tau = 3 → median point estimate ≈ 2.94
– True tau = 5 → median point estimate ≈ 4.92

The bias is negligible.

Interval estimation also performs at nominal levels: coverage across all three true values ranges from 94.8% to 95.1%. Methodologically, this is well-supported: systematic comparisons of nine heterogeneity variance estimators recommend restricted maximum likelihood over moment estimation, noting that the latter tends to be systematically biased low in small samples and rare outcomes (PMID 30067315, Langan et al., Res Synth Methods 2019); comparisons of sixteen methods further advise reporting both intervals and point estimates together (PMID 26332144, Veroniki et al.). While the Q-profile method for intervals is standard, Q-statistic-based methods yield coverage noticeably lower than nominal levels under practical conditions (PMID 30589219, van Aert et al.), making the 95% here an optimistic upper bound.

Conclusion: Estimating tau is straightforward. Thirty groups of thirty cases each are sufficient.

3. The real difficulty lies in deciding “over or under the line,” and benign vs. harmful asymmetry

This is the paper’s core finding.

When judging against the 3.59 threshold, requiring the entire interval to fall on one side to declare a decision yields:
– True tau = 2 (ban actually too strict, error rate only 1.2%): correctly identified in 73.8% of cases, but one-quarter are misclassified as “ban holds.”
– True tau = 5 (ban truly valid, error rate 26.7%): correctly identified in 96.1% of cases, with only 3.9% misclassification.

The asymmetry is stark: 74% vs. 96%. The reason is straightforward: declaring “too strict” requires the upper bound to be below the threshold, and upper bounds are inherently wider—median upper bounds are 1.44 times the point estimate with 30 groups. Declaring “valid” merely requires the point estimate to exceed the threshold.

Thus, this procedure is lenient toward “maintaining the ban” but strict toward “overturning it.” The burden of proof falls on the side seeking to overturn, and it is imposed unconsciously. This contrasts with common statistical conservatism, which usually means “not claiming effectiveness”; here, it means “not claiming no problem.”

Crucially, the region just above the threshold is nearly impossible to resolve. When true tau equals exactly 3.59, 94.9% of samples fall into a gray zone—neither clearly too strict nor clearly valid. This interval is precisely where answers matter most.

4. Adding cases helps little; the bottleneck is the number of groups

Intuition suggests more cases improve tau estimation. Calculation shows otherwise.

Fixing 30 groups and increasing cases per group from 30 to 1,920 (64-fold), the standard error of mu drops only from 0.76 to 0.66 percentage points. This occurs because tau² accounts for 75% of total variance initially, rising to 99.5%—noise is suppressed, leaving only true heterogeneity, which can only be averaged out by increasing the number of groups.

To shrink the “error rate interval width” below 10 percentage points:
– 30 cases per group → need 80 groups
– 60 cases per group → need 50 groups

To achieve 95% confidence in declaring “benign”:
– True tau = 2 → requires 80 groups
– True tau = 3 → requires 500 groups
– True tau = 3.4 → requires more than 800 groups

The closer to the threshold, the harder it is to decide. This is not a precision issue but a structural limitation of the decision rule.

5. Small samples can disguise benign cases as perfect

A further complication arises from small K. Classical moment estimation is truncated at zero when K is small: with true tau = 1 and K = 10, 40% of samples have tau estimated as zero. Zero implies “complete agreement across groups, no cross-scenario fluctuation”—the ideal state.

This creates reverse distortion: the most benign, least variable data are disproportionately reported as “zero fluctuation.” Practically, zero is read as “this ban is entirely unnecessary.”

6. What can be given and what cannot

What can be given: A retrospective of 30 to 80 groups, each with 30 cases, can convert tau from a hypothetical parameter into an interval-based measured value, with median estimates nearly unbiased. This is feasible.

What cannot be given: Using the same retrospective to decide “should the ban remain or be abolished.” Near the threshold, decisions are unattainable; benign cases are misclassified in one-quarter of instances; adding cases does not resolve this.

Correct usage: Do not ask “Did tau exceed 3.59?” Instead, ask “Where does the interval of mu / tau lie?” and report this ratio rather than the absolute value of tau. With 80 groups, the ratio interval ranges from 1.06 to 1.86, corresponding to error rates of 25% to 6%. Even this wide interval avoids borrowing units and drifting with mu.

Evidence and Limitations

Verified: Moment estimation yields nearly unbiased point estimates for small-to-moderate K, and Q-profile intervals achieve coverage close to nominal levels; these findings are consistent with this paper’s simulations and methodological literature (PMID 30067315, PMID 26332144). Restricted maximum likelihood outperforms moment estimation, which is consensus.

Partially verified: The asymmetry that “benign cases are harder to identify than harmful ones” is stably reproduced in this simulation (74 vs. 96 with 30 groups), but it depends on the decision rule requiring the entire interval to fall on one side. Alternative rules (e.g., using only point estimates) eliminate asymmetry at the cost of control. Since no external standard exists for the decision rule itself, this is a design choice of this paper.

To be verified:
1. Whether mu and tau are independent in real judgment data—if they are correlated (e.g., severe central fluctuation implies large mu), the distribution of the ratio-based criterion would distort; this paper assumes independence.
2. Whether historical data with 30 groups can be found in literature—most consistency reports aggregate kappa rather than reporting group-level differences, making reconstruction of K unlikely.
3. Whether the simulation assumption of equal cases and noise per group holds—real data often vary both, requiring separate weighting validation.

Related Reading

Evidence level: C
References: PMID 30067315, PMID 26332144, PMID 30589219, PMID 28122727
L1 Network Status: CORRESPONDENCE_NETWORK.md (1,956 lines) word-count: “heterogeneous” 0 times, “random effects” 0 times, “tau” 0 times, “error rate” 0 times, “kappa” 0 times, “variance” 1 time (line 1,694, E-005 sound rhythm entry, referring to fundamental frequency variance, an acoustic measure unrelated to cross-scenario variance components), “bias” 1 time (line 265, KB-338 self-reported narrative review without bias assessment), “rater” 4 times, “scorer” 1 time, “judgment” 1 time (line 1,832, E-061 driving direction judgment requires cross-family methods), all falling within RSA reading specifications and acupoint control arm lists. EDGE_TABLE_science-yixue.md (63 lines) grep for “kappa / reliability / consistency / variance / error / bias / calibration / heterogeneous / random effects” yields zero hits. The closest edge is E-060 label premise (organ differentiation as grouping labels | premise | label reliability, status conflict unresolved)—KB-317 records two mismatches, KB-333 notes ordinal classification requires coefficient selection and estimation precision to be tiered, KB-334 demands AC2 and prevalence reporting separately; this edge only registers “is label reliability sufficient,” not “how large is cross-scenario heterogeneity and can it be measured.” This topic has not yet entered the L1 corresponding network, registered instead in science-yixue-l1-gaps.md (cluster 60 of “measurement consistency”).


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *