融合模型赢的那一局,是赢在信息还是赢在挑选

Written by

in

Language

Evidence level: B

当”融合模型”和两个单模态模型在同一份数据上一起训练、一起评估、再挑最好的那个报出来,它赢的多半不是信息,而是挑选。

正文

一、”双模态更好”这个结论几乎是必然的

先做一个不需要生物学的推演。假设 HRV 对辨别穴位完全无用——它携带的是纯噪声。你手上仍有三个模型:fMRI 单用、HRV 单用、两者拼接。在同一份交叉验证里,三者 AUC 都是随机变量,围绕真值各自波动,而你报的是最大值。即使三者真值完全相同,最大值也必然高于平均——这是次序统计量的性质,与”有没有增益”无关。

Varma 与 Simon 说得很清楚:用交叉验证误差挑选模型、再把这个误差当作该模型的性能报告,估计是有偏的(PMID 16504092),方向稳定偏乐观。所以”双模态 AUC 高于单模态”这一观察本身不构成证据。它需要一个正确的零假设:不是 AUC=0.5,而是”三个模型在同一份数据上取最大值”会得到什么。

麻烦在多分类里会被放大。五个穴位、理论随机水平 20%,但 Combrisson 与 Jerbi 指出,小样本下真正的偶然水平不是 1/k,会随样本量与类别分布一起上移(PMID 25596422)。你以为的”显著高于随机”,可能连基准都设错了。

二、两个模态不是两桶可以相加的水

融合的直觉是”信息互补”。互补需要两个模态是同一目标的不同侧面,且彼此不重复。

HRV 与 BOLD 恰恰不满足后者。Chang 等人的分析显示,心率变异性与静息态功能连接的波动本身相关(PMID 23246859)——自主神经的慢波动会同时写进心跳间隔和脑血管信号。两条通道共享一部分方差,融合时被计入两次。Sui 等人的综述因此强调,多模态融合的方法学核心不是把特征拼起来,而是建模模态之间的协方差结构(PMID 22108139)。拼接是最粗暴的做法,其”增益”高度依赖特征尺度——fMRI 十万维、HRV 个位数维,不加权时融合约等于”只用 fMRI”,加权就多出一个待调参数,而这个参数恰好会往”让结果更好看”的方向调。

更关键的是共同原因。若某些穴位刺进去更疼、更得气,被试整体唤醒更高,融合就只是把”谁更紧张”测了两遍。AUC 会上升,特异性不会。这时”双模态更优”翻译过来是:我们更擅长识别被试的唤醒状态。

三、第二根支柱的地基比第一根软

源文把 HRV 当第二根支柱,主要指标是低频/高频比。但 Billman 的批评很直接:LF/HF 并不准确测量交感-迷走平衡(PMID 23431279)——它受呼吸频率、总功率、体位影响,把双变量问题压成一个比值。

时间尺度的问题更具体。HRV 国际标准把”短时”定为 5 分钟,低频成分需至少 2 分钟才可解释(PMID 8737210)。而任务态 fMRI 一个刺激段常只有几十秒到两三分钟——从这段记录算出的频域指标,已经不是标准定义的那个东西。2026 年一项验证研究指出,超短时 HRV 的效度必须按状态分别建立,状态一变一致性就掉(PMID 42243172)。针刺恰恰是”状态在变”的场景。

一根地基较软的支柱接进融合模型,期望增益不会大于零——它更可能贡献噪声,而噪声恰好能通过选择偏倚变成”提升”。

四、真正该做的检验,是把”挑选”也当成一个模型

判据不是”融合 > 单模态”,而是”融合的增益 > 挑选本身带来的增益”。五条可执行:

① 预注册三模型。把融合定为主分析,另两个只作敏感性分析——不让数据决定谁上场。

② 嵌套交叉验证。外层只做评估,内层做特征选择与融合权重。Kriegeskorte 的”双重 dipping”警告在此尤其致命——融合给了你更多”看着结果挑”的自由度(PMID 19396166)。

③ 置换检验重跑全流程。打乱标签后,特征选择、加权、模型选择全部重做,得到”max-of-three”的零分布;融合增益须超过该零分布 95 分位才算成立。只重训一次的置换检验,测的是另一个问题。

④ ΔAUC 用配对比较。DeLong 方法适用于同一批受试者上的相关 ROC 曲线(PMID 3203132),但它假定模型已固定;模型是训练出来时方差被低估,须改用嵌套交叉验证的配对折差做 bootstrap。

⑤ 先算功效再看结果。小样本下交叉验证的误差条宽到足以吞掉模型间差异(PMID 28655633);脑-行为关联要稳定复现,样本需以千计(PMID 35296861)。以 40-60 人、3-5 个穴位的规模,可检出的 ΔAUC 下限远高于融合的典型增益。最可能的结局是:看得到提升,检不出差异。

五、易学那一层:参合是校验,不是拼接

《灵枢·邪气脏腑病形》给三诊各一个字:”见其色,知其病,命曰明;按其脉,知其病,命曰神;问其病,知其处,命曰工。”紧接着一句:”能参合而行之者,可以为上工。”

这段话常被引来证明”古人早就主张多模态融合”,但它给的是”上工”这个身份判断,不是”参合准确率更高”这个数字。
更重要的是古人”参合”的操作方式:校验。色与脉一致则确信;相反则再问、再按,甚至判断”舍脉从症”还是”舍症从脉”。矛盾时,结论是降信。

特征拼接不是这样运作的。矛盾时它给出加权平均——一个既不是 A 也不是 B 的中间值,还往往带着比单模态更高的置信度。这才是最危险的地方:它不区分”互相印证”和”互相掩盖”。

所以要问的不是”两通道加起来准不准”,而是”两通道打架时,模型会不会告诉你它在打架”。

证据与局限

已验证: ①交叉验证选模型后直接报告该误差会产生乐观偏倚(PMID 16504092);②小样本多分类的经验偶然水平显著高于理论 1/k(PMID 25596422);③先选特征再做交叉验证构成循环分析(PMID 19396166);④HRV 与 BOLD 波动共享方差,两模态并非独立信息源(PMID 23246859);⑤LF/HF 不能准确度量交感-迷走平衡(PMID 23431279);⑥HRV 频域指标的短时定义有明确时长门槛(PMID 8737210)。

部分验证: 多模态融合在精神疾病分类中确有增益报告(PMID 23108278),但增益幅度普遍温和、且多为单队列结果;融合方法学综述强调需建模模态协方差而非简单拼接(PMID 22108139);小样本下机器学习验证需嵌套设计(PMID 31697686);针灸 fMRI 的 MVPA 判别研究已证明”可解码”,但均未做单/双模态增量对比(PMID 20071124)。

待验证: ①穴位任务态 fMRI+HRV 的双模态增益是否超过 max-of-three 零分布;②融合增益在控制得气强度、疼痛评分、平均心率后是否仍存在;③HRV 频域指标在 1-3 分钟刺激段内的重测信度是否足以支撑其作为独立判别通道;④”参合式校验”(矛盾即降置信)与”拼接式融合”在同一批数据上的判别力与校准差异。

相关阅读

证据等级: B
参考文献: PMID 16504092, PMID 25596422, PMID 19396166, PMID 23246859, PMID 22108139, PMID 23108278, PMID 23431279, PMID 8737210, PMID 42243172, PMID 3203132, PMID 28655633, PMID 35296861, PMID 31697686, PMID 20071124


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

The Winning Move of the Integrated Model: Information or Selection?

Evidence level: B

When a “fusion model” is trained, evaluated, and selected alongside two unimodal models on the same dataset, its victory is likely not due to information gain, but rather to selection bias.

Main Text

I. The Conclusion That “Multimodal is Better” Is Almost Inevitable

First, let us perform a deduction that requires no biological assumptions. Assume HRV is entirely useless for identifying acupoints—it carries only pure noise. You still have three models: fMRI alone, HRV alone, and their concatenation. In the same cross-validation split, their AUCs are random variables fluctuating around their true values, and you report the maximum. Even if all three true values are identical, the maximum must necessarily exceed the average—this is a property of order statistics, unrelated to whether there is actual gain.

Varma and Simon state clearly: selecting models via cross-validation error and then reporting that error as the model’s performance is biased (PMID 16504092), with a stable direction of overestimation. Therefore, the observation that “multimodal AUC exceeds unimodal” does not constitute evidence on its own. It requires a correct null hypothesis: what would the maximum yield if three models were selected from the same dataset?

The issue is amplified in multi-class settings. With five acupoints and a theoretical random baseline of 20%, Combrisson and Jerbi point out that the true chance level under small samples is not 1/k but shifts upward depending on sample size and class distribution (PMID 25596422). What you perceive as “significantly above random” may simply be based on a misconfigured baseline.

II. Two Modalities Are Not Two Buckets of Water That Can Be Added

The intuition for fusion is “information complementarity.” Complementarity requires two modalities to represent different aspects of the same target without redundancy.

HRV and BOLD fail this latter condition. Chang et al.’s analysis shows that heart rate variability and resting-state functional connectivity fluctuations are inherently correlated (PMID 23246859)—slow fluctuations in the autonomic nervous system are simultaneously encoded in heart intervals and cerebrovascular signals. Two channels share variance; when fused, this is counted twice. Consequently, Sui et al.’s review emphasizes that the methodological core of multimodal fusion is not concatenating features, but modeling the covariance structure between modalities (PMID 22108139). Concatenation is the most crude approach; its “gain” heavily depends on feature scaling—fMRI has tens of thousands of dimensions, while HRV has only a few. Without weighting, fusion approximates “using only fMRI”; adding weights introduces an extra hyperparameter that tends to be tuned toward making the results look better.

More critically, there are common causes. If certain acupoints cause more pain or a stronger sensation of “qi” upon insertion, leading to higher overall arousal in subjects, fusion merely measures “who is more tense” twice. AUC rises, but specificity does not. In this case, “multimodal superiority” translates to: we are better at identifying the subject’s arousal state.

III. The Foundation of the Second Pillar Is Softer Than the First

The source text treats HRV as the second pillar, primarily using the Low-Frequency/High-Frequency ratio. However, Billman’s critique is direct: LF/HF does not accurately measure sympathetic-vagal balance (PMID 23431279)—it is influenced by respiratory rate, total power, and posture, compressing a bivariate problem into a ratio.

The issue of time scale is more specific. The HRV international standard defines “short-term” as 5 minutes, requiring at least 2 minutes to interpret low-frequency components (PMID 8737210). In contrast, task-state fMRI stimulus segments often last only tens of seconds to a few minutes. Frequency domain metrics derived from such short recordings are no longer the standard-defined quantities. A 2026 validation study indicates that the validity of ultra-short HRV must be established separately for each state, as consistency drops when the state changes (PMID 42243172). Acupuncture is precisely a scenario where the state is changing.

A pillar with a softer foundation enters the fusion model; expected gain will not exceed zero—it is more likely to contribute noise, and noise can恰好 (coincidentally) become an “improvement” through selection bias.

IV. The Real Test Should Treat “Selection” as a Model

The criterion is not “fusion > unimodal,” but rather “fusion gain > gain from selection itself.” Five actionable steps:

① Pre-register three models. Define fusion as the primary analysis and the other two only for sensitivity analysis—do not let the data decide who enters.

② Nested cross-validation. The outer loop performs evaluation only; the inner loop handles feature selection and fusion weighting. Kriegeskorte’s warning against “double dipping” is particularly fatal here—fusion grants you more freedom to “look at results and select” (PMID 19396166).

③ Permutation test re-running the full pipeline. Shuffle labels and re-run feature selection, weighting, and model selection to obtain the zero distribution for “max-of-three”; fusion gain must exceed the 95th percentile of this zero distribution to be considered valid. A single retraining permutation test measures a different question.

④ Use paired comparison for ΔAUC. The DeLong method applies to correlated ROC curves on the same subjects (PMID 3203132), but it assumes fixed models; if model variance is underestimated during training, use nested cross-validation paired differences with bootstrap.

⑤ Calculate power before results. Under small samples, the width of cross-validation error bars is sufficient to swallow inter-model differences (PMID 28655633); brain-behavior associations require thousands of samples for stable replication (PMID 35296861). With a scale of 40-60 subjects and 3-5 acupoints, the detectable lower limit of ΔAUC far exceeds typical fusion gains. The most likely outcome is: visible improvement, but no detectable difference.

V. The Yijing Layer: Parametric Combination is Verification, Not Concatenation

The Ling Shu chapter “Pathogenic Qi and Disease Forms of Organs” assigns one character to each of the three diagnostics: “See its color, know its disease; this is called ‘Ming’ (Clear); Press its pulse, know its disease; this is called ‘Shen’ (Spirit); Ask about the disease, know its location; this is called ‘Gong’ (Work).” Immediately following is: “Those who can combine and practice these are called ‘Shang Gong’ (Superior Practitioner).”

This passage is often cited to prove “ancients already advocated multimodal fusion,” but it provides a judgment of the identity “Shang Gong,” not a numerical claim that “combination accuracy is higher.”

More importantly, the ancient method of “parametric combination” (参合) operates as verification: if color and pulse align, confidence is confirmed; if they contradict, one asks again, presses again, or judges whether to “discard pulse for symptom” or “discard symptom for pulse.” In contradiction, the conclusion is reduced credibility.

Feature concatenation does not operate this way. In contradiction, it provides a weighted average—a midpoint that is neither A nor B—and often carries higher confidence than unimodal results. This is the most dangerous aspect: it fails to distinguish between “mutual corroboration” and “mutual masking.”

Therefore, the question should not be “how accurate are two channels added together,” but rather “when the two channels conflict, will the model tell you it is in conflict.”

Evidence and Limitations

Verified: ① Reporting cross-validation error directly after model selection produces optimistic bias (PMID 16504092); ② Empirical chance levels in small-sample multiclass classification are significantly higher than the theoretical 1/k (PMID 25596422); ③ Selecting features before cross-validation constitutes circular analysis (PMID 19396166); ④ HRV and BOLD fluctuations share variance; the two modalities are not independent information sources (PMID 23246859); ⑤ LF/HF cannot accurately measure sympathetic-vagal balance (PMID 23431279); ⑥ Short-term definitions for HRV frequency domain metrics have explicit duration thresholds (PMID 8737210).

Partially Verified: Multimodal fusion reports gains in psychiatric classification (PMID 23108278), but gain magnitudes are generally modest and mostly single-cohort results; fusion methodology reviews emphasize modeling modal covariance rather than simple concatenation (PMID 22108139); machine learning validation in small samples requires nested design (PMID 31697686); acupuncture fMRI MVPA decoding studies have proven “decodability,” but none performed unimodal/bimodal incremental comparisons (PMID 20071124).

To Be Verified: ① Whether bimodal gains in acupoint task-state fMRI+HRV exceed the max-of-three zero distribution; ② Whether fusion gains persist after controlling for sensation intensity, pain scores, and average heart rate; ③ Whether retest reliability of HRV frequency metrics within 1-3 minute stimulus segments is sufficient to support them as independent discriminative channels; ④ The difference in discriminative power and calibration between “parametric verification” (reduced confidence on contradiction) and “concatenative fusion” on the same dataset.

Related Reading

Evidence Level: B
References: PMID 16504092, PMID 25596422, PMID 19396166, PMID 23246859, PMID 22108139, PMID 23108278, PMID 23431279, PMID 8737210, PMID 42243172, PMID 3203132, PMID 28655633, PMID 35296861, PMID 31697686, PMID 20071124


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *