范围损失不该用「结论还成立的比例」来数:换成分布重叠度,反而更靠不住

Written by

in

Language

Evidence level: C

分布重叠度量的是一群人长得像不像,范围损失问的是结论搬过去还灵不灵——这两件事之间缺一整座桥。

一、这个替换想解决什么,又解决了什么

上一篇把降级损失拆成两笔账:功效损失能靠加样本补,范围损失补不回。但「补不回多少」是用「参数真值空间中结论仍成立的比例」算的。这个定义有毛病:真值空间是自己设的,模拟端填什么它就答什么,等于自出自判。

于是有了这个想法:改用修炼者与新手两群的协变量分布重叠度来度量——重叠越低,人群差越远,结论越不可靠。听上去很顺,但顺的地方往往就是坑。

二、先分清两个词:重叠是形状,损失是可用性

分布重叠系数(overlap coefficient, OVL)是两个分布密度函数逐点取小后的积分,取值 0 到 1。它是纯几何量:只描述两条曲线的交叠面积,不含任何关于「结论」的信息。

范围损失问的是另一件事:把在 A 群成立的结论拿到 B 群,方向和数值还对不对。一个变量可以两群分布天差地别(OVL 0.4),但它根本不是效应修饰变量,搬过去毫无损失;也可以两群分布几乎一样(OVL 0.95),而它恰好是强效应修饰变量,一搬就翻车。OVL 只看得见前者,看不见后者。

三、重叠系数本身并不好算

这个量在文献里长期被冷落,主因是缺软件、缺覆盖全分布情形的推断方法。近年才有系统处理:参数法(二正态模型)在大范围分布情形下优于非参数法,并提供 R 代码(PMID 34693817);另有工作把它扩展为「协变量特异」版本,用线性回归加可能的 Box-Cox 变换估计,自助法构造置信区间(PMID 40852736)。

后一篇的关键含义:OVL 本身可以被协变量改写。年龄一变,重叠系数就变。那么把「某变量上的 OVL」当作固定的人群差指标,前提就已经不成立。

四、更要紧的是:它不是新信息

在二正态假设下,OVL 与曲线下面积(AUC)之间存在确定的单调对应——知道一个就能算出另一个。这意味着 OVL 并不携带判别力之外的独立信息,只是 AUC 的一个非线性重标。

把一个本质上是判别力的量改名叫「可推广性」,正是这类口径替换最典型的自欺:数字变了,含义没变。判别力回答「能不能分开两类人」,可推广性回答「换个群体结论还对不对」——后者还取决于效应修饰结构,那部分信息 OVL 里一个比特都没有。

五、加权合成那一步最脆

第二步是「重叠系数 × 该变量对效应量的调节幅度」再加权合成。这里有两个独立的塌方点。

其一,调节幅度估不准。在按总体效应设计的试验里,交互检验的功效远低于总体检验:总体效应有 80% 功效的试验,检出同量级交互只有约 29% 功效;要同等功效,样本量须扩到四倍(PMID 15066682)。亚组分析假阳性与假阴性的量级也早有量化(PMID 11701102)。乘子里最不确定的那一项,恰好是权重最大的一项。

其二,量纲不可通约。重叠系数是无量纲比例,调节幅度是效应量单位,两者相乘得到的指数没有物理含义,只能内部比较——它答不了「损失百分之几」,只能答「比另一组大还是小」。

六、单变量重叠不等于联合重叠

多个协变量各自重叠 0.9,联合起来可能高度不重叠,这是高维几何的常识。而多变量的「重叠系数」没有唯一定义:逐点取小在高维仍可写,但估计方差爆炸;改用能量距离或核两样本检验,得到的是检验统计量而非 0-1 比例,不能乘调节幅度。

更要紧的是方向性。真正的可推广性识别条件要求「参与×处理」在绝对或相对尺度上无交互(PMID 40266689),这是关于结构的主张,不是关于分布的描述;后分层与 g 方法能借目标人群协变量加权(PMID 41992116),前提仍是效应修饰变量集合正确。分布差异只是必要条件的一个代理,从来不是充分条件。

七、反转:不问损失多大,问损失落在谁身上

与其造一个合成指数,不如把问题改成可执行的三问:

其一,先做平衡诊断。标准化均值差、方差比逐变量报(PMID 30788363),别急着压成一个数。其二,把交互检验的功效算出来写进方案——若某变量的交互在现有样本下检出功效不足 30%(PMID 15066682),该变量的「调节幅度」应标为不可估,而不是填个点估计去乘。其三,报告适用范围时只报「本研究人群的协变量区间」,不报「可外推全体」——社区抽样与试验入选标准的落差可以大到只有中位 4% 的患者符合条件(PMID 17105779)。

判据也据此改写:若逐变量诊断显示关键效应修饰变量标准化均值差 <0.1,则降级过度保守,可保留合并;若 >0.25 且该变量交互功效 >80%,则降级必要,应按该变量分层报告;若交互功效 <30%,则无论重叠多高,都只能报「该变量调节幅度未知、结论适用范围未经检验」——这时任何合成指数都是把未知数乘上一个系数,得到的仍是未知数。

证据与局限

已验证: OVL 的推断方法学已系统建立,参数法(二正态)在大范围分布情形下优于非参数法,已有 R 实现(PMID 34693817);协变量特异 OVL 可用回归加 Box-Cox 估计、自助法给置信区间(PMID 40852736);按总体效应设计的试验对交互检验功效严重不足,同量级交互仅约 29% 功效、需四倍样本(PMID 15066682);亚组分析的假阳性与假阴性风险已量化(PMID 11701102);倾向得分匹配后的平衡诊断应以标准化均值差与方差比逐变量报告(PMID 30788363);试验入选标准与社区人群的落差可量化,哮喘领域仅中位 4% 符合(PMID 17105779);从试验外推需「参与×处理」无交互的结构性假设(PMID 40266689),后分层可借目标人群协变量加权(PMID 41992116)。

部分验证: 效应修饰与不可压缩性并存时,边际估计与条件估计可给出方向相反的治疗决策(PMID 41626969)——这支持「口径选择本身会改变结论」,但其结论建立在二值结局与比数比尺度上,连续型丹田参数是否同样敏感尚未检验;无正性假设时重叠权重法把目标人群限定为「有足够重叠者」(PMID 38847059),思路与本文一致,但那是估计处理效应的方法,不是度量范围损失的方法;加法与乘法尺度下交互的度量不可互换(PMID 16620385),提示「调节幅度」必须先声明尺度。

待验证: 「加权范围损失指数」在丹田五参数上从未算过,其与「结论成立比例」口径的差值方向与量级均无数据;OVL 与 AUC 在二正态下的对应关系是否可外推到实际偏态的丹田参数分布,尚未核;修炼者与新手两群在候选效应修饰变量上的重叠系数实测值(提案预期普遍 >0.8)纯属假设;若实测确认普遍 >0.8,则两笔账的二分在本项目内失效、须整条撤回——这一条本身就是本篇最重要的可证伪点。

相关阅读

证据等级: C
参考文献: PMID 40852736, PMID 34693817, PMID 15066682, PMID 11701102, PMID 30788363, PMID 17105779, PMID 40266689, PMID 41992116, PMID 41626969, PMID 38847059, PMID 16620385
L1 网络状态:该主题尚未进入 L1 对应网络——CORRESPONDENCE_NETWORK.md 节点表按易学/中医侧、物理侧、生理/体验侧三类域分派(原文「| 节点 | 各域的概念/量(五行、五脏、频率、颜色…) | 按域分派(见下表) |」),三类域均无「范围损失 / 效应修饰变量 / 协变量分布重叠 / 目标人群」这类测量学与外效度维度节点;边表止于 E-050(「六十甲子纳音 | 对应 | 60 个频率」,层级 L1、状态 open,现代证据栏原文「尚无任何可测来源」),无任何以分布重叠、效应修饰或可推广性为源或目标的边。EDGE_TABLE_science-yixue.md 为反抽统计,仅列概念计数(丹田 51 篇 / 气功 48 篇 / 呼吸 189 篇 / 频率 407 篇)与最高证据级,无 L1/L2/L3 层级与 active/open/conflict 状态判定,故本条边状态为缺。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Range loss should not be counted by \”proportion of valid conclusions\”: using distribution overlap is even less reliable

Evidence level: C

Distribution overlap measures how similar a group of people looks; range loss asks whether a conclusion remains valid when transferred to another group—there is a missing bridge between these two concepts.

I. What This Replacement Aims to Solve, and What It Actually Solves

The previous post decomposed degradation loss into two accounts: efficacy loss can be compensated by increasing sample size, but range loss cannot. However, “how much cannot be compensated” was calculated using the “proportion of conclusions that still hold within the true value space.” This definition has flaws: the true value space is self-defined, and the simulation endpoint answers whatever it is fed, amounting to self-validation.

Thus arose this idea: use the overlap of covariate distributions between practitioners and novices to measure it—lower overlap implies greater population difference and less reliable conclusions. It sounds smooth, but smoothness often hides pitfalls.

II. First Distinguish Two Terms: Overlap Is Shape, Loss Is Usability

The overlap coefficient (OVL) is the integral of the pointwise minimum of two distribution density functions, ranging from 0 to 1. It is a purely geometric quantity: it describes only the intersection area of two curves and contains no information regarding “conclusions.”

Range loss asks about a different matter: whether the direction and magnitude of a conclusion valid in Group A remain correct when applied to Group B. One variable may have vastly different distributions between two groups (OVL 0.4), yet not be an effect modifier at all, resulting in no loss upon transfer; conversely, two groups may have nearly identical distributions (OVL 0.95), yet the variable happens to be a strong effect modifier, causing failure upon transfer. OVL sees only the former, not the latter.

III. The Overlap Coefficient Itself Is Difficult to Calculate

This quantity has long been neglected in the literature primarily due to a lack of software and inferential methods covering all distribution scenarios. Systematic handling has only recently emerged: parametric methods (two-normal models) outperform non-parametric methods in broad distribution scenarios and provide R code (PMID 34693817); other work extends it to a “covariate-specific” version using linear regression with potential Box-Cox transformations for estimation and bootstrap methods to construct confidence intervals (PMID 40852736).

The key implication of the latter post is: OVL itself can be rewritten by covariates. If age changes, the overlap coefficient changes. Therefore, treating “OVL on a specific variable” as a fixed metric of population difference presupposes a condition that is already invalid.

IV. More Importantly: It Is Not New Information

Under the assumption of bivariate normality, OVL has a deterministic monotonic correspondence with the area under the curve (AUC)—knowing one allows calculation of the other. This means OVL does not carry independent information beyond discriminative power; it is merely a non-linear rescaling of AUC.

Renaming a quantity that is essentially about discriminative power as “generalizability” is a classic form of self-deception in such metric substitutions: the number changes, but the meaning does not. Discriminative power answers “Can we distinguish two types of people?” while generalizability asks “Is the conclusion still correct in a different population?”—the latter depends on effect modification structures, information of which OVL contains zero bits.

V. The Weighted Synthesis Step Is Most Fragile

The second step is “overlap coefficient × the magnitude of modulation by that variable on the effect size,” then weighted synthesis. Here lie two independent collapse points.

First, modulation magnitude is estimated inaccurately. In trials designed for overall effects, the power for interaction tests is far lower than for overall tests: a trial with 80% power for overall effects detects interactions of similar magnitude with only about 29% power; to achieve equivalent power, sample size must be quadrupled (PMID 15066682). The magnitudes of false positives and false negatives in subgroup analysis have also been quantified for some time (PMID 11701102). The least certain term in the multiplier is precisely the one with the largest weight.

Second, units are incommensurate. The overlap coefficient is a dimensionless ratio, while modulation magnitude is in effect size units; their product yields an index with no physical meaning, usable only for internal comparison—it cannot answer “what percentage of loss,” only “larger or smaller than another group.”

VI. Univariate Overlap Does Not Equal Joint Overlap

With multiple covariates, each having an overlap of 0.9, they may jointly be highly non-overlapping—a common knowledge of high-dimensional geometry. Furthermore, the “overlap coefficient” for multivariate data lacks a unique definition: taking the pointwise minimum in high dimensions can still be written, but estimation variance explodes; using energy distance or kernel two-sample tests yields a test statistic rather than a 0-1 proportion, which cannot be multiplied by modulation magnitude.

More importantly is directionality. True conditions for generalizability identification require no interaction between “participation × treatment” on absolute or relative scales (PMID 40266689)—a claim about structure, not description of distributions; post-stratification and the g-method can weight by target population covariates (PMID 41992116), but only if the set of effect modifiers is correct. Distribution differences are merely a proxy for a necessary condition, never a sufficient one.

VII. Inversion: Do Not Ask How Large the Loss Is, But Where It Falls

Rather than constructing a synthetic index, reframe the problem into three executable questions:

First, conduct balance diagnostics. Report standardized mean differences and variance ratios variable by variable (PMID 30788363); do not rush to compress them into a single number. Second, calculate the power for interaction tests and incorporate it into the protocol—if the interaction for a specific variable has insufficient detection power (<30%) under current sample size (PMID 15066682), mark that variable’s “modulation magnitude” as unestimable rather than filling in a point estimate to multiply. Third, when reporting applicability, report only the “covariate range of this study population,” not “extrapolable to all”—the gap between community sampling and trial inclusion criteria can be so large that only the median 4% of patients meet eligibility (PMID 17105779).

Criteria are also rewritten accordingly: if variable-by-variable diagnostics show standardized mean differences for key effect modifiers <0.1, degradation is overly conservative, and pooling may be retained; if >0.25 and interaction power >80%, degradation is necessary, and stratification by that variable should be reported; if interaction power <30%, regardless of how high the overlap is, only report “modulation magnitude for this variable unknown, applicability of conclusions not verified”—at this point, any synthetic index multiplies an unknown by a coefficient, yielding still an unknown.

Evidence and Limitations

Verified: Inferential methodologies for OVL are systematically established; parametric methods (two-normal) outperform non-parametric methods in broad distribution scenarios with R implementation available (PMID 34693817); covariate-specific OVL can use regression with Box-Cox estimation and bootstrap for confidence intervals (PMID 40852736); trials designed for overall effects suffer severely from insufficient power for interaction tests, with similar interactions having only ~29% power and requiring quadrupled sample sizes (PMID 15066682); false positive and false negative risks in subgroup analysis have been quantified (PMID 11701102); balance diagnostics after propensity score matching should report standardized mean differences and variance ratios variable by variable (PMID 30788363); the gap between trial inclusion criteria and community populations is quantifiable, with only the median 4% eligible in asthma domains (PMID 17105779); extrapolation from trials requires structural assumptions of no interaction between “participation × treatment” (PMID 40266689), and post-stratification can weight by target population covariates (PMID 41992116).

Partially Verified: When effect modification and incompressibility coexist, marginal and conditional estimates can yield opposite treatment decisions (PMID 41626969)—supporting the notion that “metric choice itself alters conclusions,” but its conclusions are built on binary outcomes and odds ratio scales; whether continuous-type丹田 parameters are similarly sensitive remains untested; without positivity assumptions, overlap weighting methods restrict the target population to “those with sufficient overlap” (PMID 38847059), aligning with this paper’s思路 but serving as an estimation method for treatment effects rather than a metric for range loss; interaction metrics are not interchangeable under additive and multiplicative scales (PMID 16620385), suggesting “modulation magnitude” must first declare its scale.

To Be Verified: The “weighted range loss index” has never been calculated on丹田 five parameters; the direction and magnitude of difference between this metric and the “proportion of conclusions holding” lack data; whether the correspondence between OVL and AUC under bivariate normality can be extrapolated to actual skewed丹田 parameter distributions remains unverified; measured OVL values between practitioners and novices on candidate effect modifiers are purely hypothetical (proposal expects generally >0.8); if measured confirmation shows generally >0.8, the binary accounting within this project fails and must be withdrawn entirely—this point itself is the most falsifiable aspect of this paper.

Related Reading

Evidence Level: C
References: PMID 40852736, PMID 34693817, PMID 15066682, PMID 11701102, PMID 30788363, PMID 17105779, PMID 40266689, PMID 41992116, PMID 41626969, PMID 38847059, PMID 16620385
L1 Network Status: This topic has not yet entered the L1 corresponding network—the CORRESPONDENCE_NETWORK.md node table assigns nodes by Yijing/TCM side, physics side, and physiological/experiential side (original text “| Node | Concept/Quantity in each domain (Five Phases, Five Organs, Frequency, Color…)| Assigned by domain (see table below)|”), with no nodes in any of the three domains measuring “range loss / effect modifier / covariate distribution overlap / target population”; edge tables stop at E-050 (“60 Heavenly Stems and Earthly Branchs Na Yin | Corresponds | 60 Frequencies”, Level L1, status open, modern evidence column originally “No measurable sources at all“), with no edges sourced or targeted by distribution overlap, effect modification, or generalizability. EDGE_TABLE_science-yixue.md is a reverse statistical count, listing only concept counts (丹田 51 papers / Qigong 48 papers / Breathing 189 papers / Frequency 407 papers) and highest evidence level, lacking L1/L2/L3 hierarchy and active/open/conflict status determination; thus, the status of this edge is missing.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *