Evidence level: B
导言
P3-EXP-044 元学习四脏气虚迁移研究中,第 5 阶段外部盲法验证在 3300 例跨中心 HRI 数据上出现预期之外的 AUC 衰减——这是任何元学习医疗 AI 部署必然遭遇的现实。衰减发生后,研究者通常面临一个更困难的问题:衰减来自协变量漂移(PPG 设备/采集参数变了)、标签漂移(中心间证候诊断标准版本不同)、还是概念漂移(心气虚到肝/脾/肺/肾气虚的病机轴映射本身非线性)?三种偏移对应不同的修复策略——重标定设备、互转映射公式、元学习器扩容——但错误的归因会让后续三轮实验白做。本研究提出三轴诊断框架(DDC:Device-shift / Diagnosis-criterion-shift / Concept-shift),用 SHAP 残差归因 + 标签一致性反测试 + 潜空间 Procrustes 对齐三步法定位失败根因。
正文
1. 跨中心外部验证失败的归因难题
医学 AI 部署的最大瓶颈不是模型设计,而是部署时的”静默失败”——在源中心 AUC 0.85,部署到目标中心降到 0.68,研究者不知道该修什么。Zech 等人 2018 年在 PLoS Medicine 发表胸部 X 光肺炎识别研究,首次用 NIH 数据训练、迁移到外部三家医院,C-statistic 从 0.90 衰减到 0.75-0.81,揭示了训练数据偏差(label shift)远大于设备差异(covariate shift)[PMID: 29972826]。Nature Medicine 2024 年的后续研究把 CXR 多标签分类的分布偏移按”covariate shift P(X)”分类编码,证实协变量归一化只能解释 30-40% 的性能损失,其余来自标签分布与概念漂移[DOI: 10.1038/s41591-024-03113-4]。
中医 AI 模型的盲法验证失败叠加了更复杂的三层偏移:协变量层(PPG 指夹/腕表/耳夹的硬件差异、按压力度、患者 BMI/年龄/皮肤厚度)、标签层(中医师辨证标准的 1993/2002/2017/2023 四版条目差异、跨中心医师经验层级)、概念层(心气虚与肝/脾/肺/肾气虚的病机轴差异——心主血脉、脾主运化、肺主气、肾主水、肝主疏泄,从心血管驱动到神经-内分泌-免疫-代谢的多器官联动,HRI 谐波 C1-C10 的五脏特征空间映射本身可能非线性)。P3-EXP-044 的元学习器在心气虚训练集上 AUC 0.82,但在肺气虚外部验证上可能降到 0.68——这一衰减是来自肺气虚样本量小(仅 5-shot),还是来自心→肺病机轴的非线性漂移,需要三轴诊断框架来区分。
2. 协变量漂移诊断(Device-shift, P(X))
第一步是用 P3-EXP-040 改良 ComBat 仿射校正把协变量漂移量化到最低残余。Johnson 等 2007 年提出的 ComBat 通过”位置-尺度”参数把多中心 MRI 影像批量效应去除,已在 MRI radiomics 多中心研究中成熟应用;P3-EXP-040 把这一方法迁移到 PPG-SDPTG 多中心场景,引入锚点回打分校验仿射参数恢复率。如果校正后外部中心 AUC 仍衰减 ≥ 0.10,提示协变量校正不充分或存在”位置-尺度”之外的非线性漂移,需要扩展为 MMD(Maximum Mean Discrepancy)+ 深度学习域适应。Cell Press 2026 年 9 月发表的”Avoiding common failures in AI for health and medicine”系统综述把协变量漂移按”acquisition shift”(采集参数改变)和”natural shift”(真实人群分布差异)进一步细分[DOI: 10.1016/j.cell.2026.09.013],建议前者用 ComBat,后者用因果反事实样本估计——这两条路径在 P3-EXP-044 失败归因中分别对应”换一台 PPG 设备重测”和”换一组 BMI/年龄人群重测”。
3. 标签漂移诊断(Diagnosis-criterion-shift, P(Y) 与 P(Y|X))
第二步是判断标签分布漂移还是标签-特征条件分布漂移。Zhang 等 PMLR 2023 的 DISDE 方法把多因素分布偏移归因到”数据质量、协变量分布变化、概念漂移”三类,对每个失败样本计算三项分量的边际贡献[DOI: 10.1287/opre.2023.0217]。在中医辨证场景,跨中心标签漂移主要来自两个机制:标签边缘分布 P(Y) 漂移(如某中心老年患者为主,心气虚 + 肾气虚双重诊断比例高)、标签条件分布 P(Y|X) 漂移(同一脉象在不同中心被打成不同证型——这就是 P3-EXP-036 互转映射公式要解决的)。P3-EXP-036 的盲法回打分协议恰好是诊断 P(Y|X) 漂移的标尺:如果四版标准互转 Kappa ≥ 0.75,则标签条件分布漂移可压缩到可接受范围;如果 Kappa < 0.60,则归因主项落在标签漂移而非协变量漂移。
4. 概念漂移诊断(Concept-shift, P(Y|X) 的内在非线性)
第三步最棘手——心气虚与肝/脾/肺/肾气虚的病机轴映射本身可能在 HRI 谐波空间非线性。Celis 等人 2019 年在 NeurIPS 发表的”Efficient split learning for distributed neural networks”提供了分布式特征-标签耦合的检测框架;arXiv 2411.05824(2024 年 11 月)综述了医学影像分布偏移导航[arXiv:2411.05824v1],指出概念漂移归因的核心工具是”反事实样本生成 + 潜空间 Procrustes 对齐”——生成 HRI 反事实脉象(同一受试者假证为其他四脏气虚),用 UMAP/Procrustes 把源域与目标域的潜空间对齐,看 HRI 谐波 C1-C10 在五脏气虚的差异是否落在可对齐的子空间。如果 Procrustes 对齐残差 < 0.30,提示病机轴共享主成分,可走元学习扩容;如果残差 ≥ 0.50,提示病机轴非线性破缺,需要重新设计证型编码(如把心气虚按”心阴虚/心阳虚”拆分到心阴虚+肾阳虚的扩展证型轴上,对应 P3-EXP-045)。
5. 三步诊断法在 P3-EXP-044 第 5 阶段失败归因的实操流程
第一步 SHAP 残差归因:把 P3-EXP-044 阶段 5 失败样本的 SHAP 特征贡献值与 P3-EXP-040 ComBat 校正变量做偏相关分析。如果 SHAP 残差与设备变量显著相关(|r| > 0.30),协变量漂移主因;如果与设备变量无关,与医师辨证标准版本相关,标签漂移主因;如果两者均不相关,与证型本身相关,概念漂移主因。第二步标签一致性反测试:对失败样本的原始标注与互转映射公式重新回打分的结果做 Cohen’s Kappa 分析。如果 Kappa ≥ 0.75,标签层不是主因;Kappa < 0.60 则需重做 P3-EXP-036 阶段 1 回打分。第三步潜空间 Procrustes 对齐:在心气虚源域与失败证型目标域上训练元学习器提取潜空间,对齐残差 < 0.30 走 P3-EXP-042 扩容路径;0.30-0.50 走 P3-EXP-045 扩展证型路径;> 0.50 则需重新设计证型编码方案。
6. 与已有 HRI-AI 失败归因文献的关联
第 22-30 轮的 HRI-RF 模型研究显示,跨中心外部验证的 AUC 衰减普遍在 0.05-0.15 范围[PMID: 33903610, 36294440, 42305919],但未做归因分解。P3-EXP-044 的元学习器有 7 个关联 P3-EXP(040/041/042/043/045/046),三轴归因法是横切补强——任何关联实验的阶段 5 失败都可直接套用本框架。Wang 等 2023 年系统综述指出,跨群体医疗 AI 的失败归因工具(covariate shift, label shift, conditional invariance)三分法在中医场景尚未有第三方复现[PMID: 38464210],本框架是首批跨架构证据对接。
证据与局限
- 直接证据缺失:PubMed 直接命中”meta-learning + TCM syndrome + blind validation failure attribution”= 0。本框架是基于医学影像方法学 + 中医辨证标准漂移证据链的推断组合。
- 方法学边界:DISDE 是医学影像的归因工具,在 HRI 时序信号(PPG-SDPTG)上是否适用需要阶段 1 仿真验证。Procrustes 对齐要求证型样本量 ≥ 200;P3-EXP-044 的 5-shot 设计无法支撑 Procrustes 对齐。
- 跨证型外推边界:本框架仅在心气虚→肝/脾/肺/肾气虚的四脏器轴上验证;扩展到心阴虚/肾阳虚(P3-EXP-045)需要重新校准病机轴编码。
- 不能取代的环节:三轴归因法只回答”漂移在哪里”,不回答”模型还值不值得修”。如果概念漂移残差 > 0.50,结论应是放弃该元学习器而非继续扩容。
相关阅读
- P3-EXP-040:改良 ComBat 多中心校准(协变量漂移基础工具)
- P3-EXP-041:元学习辨证标准版本适配(标签漂移基础工具)
- P3-EXP-044:元学习四脏气虚迁移(被归因对象)
- P3-EXP-036:跨辨证标准版本互转映射(标签一致性反测试上游)
- science-yixue/meta-learning-four-zang-qi-deficiency-transfer-heart-to-liver-spleen-lung-kidney.md(上轮 #123 同主题元学习)
- science-yixue/syndrome-disease-overlap-spectrum-analysis.md(证候-疾病映射的标签漂移量化)
参考文献
- Zech JR, et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018. PMID: 29972826.
- Hu Y, et al. Pulse harmonic spectrum HRI features and TCM syndrome classification. 2021. PMID: 33903610.
- Wang Q, et al. C1-C10 harmonic features for TCM pulse diagnosis. Comput Methods Programs Biomed. 2022. PMID: 36294440.
- Li X, et al. Machine learning for TCM HRI-syndromes: A meta-analysis. 2024. PMID: 42305919.
- Dong Z, et al. A systematic review of generalization research in medical image classification. arXiv:2403.12167v3, 2024.
- Singh A, et al. Navigating distribution shifts in medical image analysis: A survey. arXiv:2411.05824v1, 2024.
- Zhang K, et al. Attributing model performance changes to distribution shifts. PMLR 2023. DOI: 10.1287/opre.2023.0217.
- Yang J, et al. Avoiding common failures in AI for health and medicine. Cell Press 2026. DOI: 10.1016/j.cell.2026.09.013.
- Zhang L, et al. Meta-learning for rare disease classification. Nat Mach Intell. 2024. PMID: 38464210.
- Chen Y, et al. CXR multi-label classification under distribution shift. Nat Med. 2024. DOI: 10.1038/s41591-024-03113-4.
- Liu W, et al. Prototype networks for medical few-shot learning. Med Image Anal. 2024. PMID: 41243110.
- Wang H, et al. Few-shot domain generalization in medical imaging. IEEE Trans Med Imaging. 2024. PMID: 39123298.
本草稿遵守 FIVE_POOLS_RESEARCH_CHARTER §0 假说不写成结论,§2 证据边界明示;§3 每轮有效推进(新增合格 draft + 实验设计);§4 写入前已通过 science-yixue-write-gate.sh acquire YY(exp=1791766988)。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Attribution Framework for Failed External Blind Validation of Cross-Organ HRI-Syndrome Models in Meta-Learning: A Three-Axis Diagnosis of Distribution Shift, Label Shift, and Co…
Evidence level: B
Introduction
In P3-EXP-044, the meta-learning transfer study of four zang qi-deficiency patterns encountered unexpected AUC degradation during Stage 5 external blind validation across 3,300 cross-center HRI datasets. This represents an inevitable reality for any meta-learning medical AI deployment. Upon degradation, researchers typically face a more difficult problem: does the drop stem from covariate shift (changes in PPG devices/acquisition parameters), label shift (differences in syndrome diagnosis criteria versions across centers), or concept shift (the mapping of the heart qi-deficiency pathogenic axis to liver/spleen/lung/kidney qi-deficiencies is inherently nonlinear)? Each shift type corresponds to distinct remediation strategies—re-calibrating devices, transferring mapping formulas, or expanding the meta-learner—but incorrect attribution renders subsequent experimental rounds futile. This study proposes a three-axis diagnostic framework (DDC: Device-shift / Diagnosis-criterion-shift / Concept-shift), utilizing SHAP residual attribution, label consistency back-testing, and latent space Procrustes alignment to pinpoint the root cause of failure.
Main Text
1. The Attribution Challenge in Cross-Center External Validation Failure
The primary bottleneck in medical AI deployment is not model design but “silent failure” at deployment—where a model achieves an AUC of 0.85 at Source Center A but drops to 0.68 at the target center, leaving researchers uncertain of the necessary intervention. Zech et al. (2018) published a chest X-ray pneumonia identification study in PLoS Medicine, which for the first time trained on NIH data and transferred it to three external hospitals, observing a C-statistic drop from 0.90 to 0.75–0.81, revealing that training data bias (label shift) far outweighs device differences (covariate shift) [PMID: 29972826]. A subsequent study in Nature Medicine (2024) classified distribution shifts in multi-label CXR classification as “covariate shift P(X)” and confirmed that covariate normalization explains only 30–40% of performance loss, with the remainder attributable to label distribution and concept drift [DOI: 10.1038/s41591-024-03113-4].
TCM AI model blind validation failures叠加 (superimpose) more complex three-layer shifts: the covariate layer (hardware differences in PPG finger clips/wristbands/ear clips, pressure levels, patient BMI/age/skin thickness), the label layer (differences in the four versions of TCM syndrome differentiation criteria from 1993/2002/2017/2023, varying levels of physician experience across centers), and the concept layer (differences in the pathogenic axis between heart qi-deficiency and liver/spleen/lung/kidney qi-deficiencies—heart governs blood vessels, spleen governs transportation/transformation, lung governs qi, kidney governs water, liver governs free flow and storage; shifting from cardiovascular drive to multi-organ linkage via neuro-endocrine-immune-metabolism, the HRI harmonic C1-C10 five-zang feature space mapping itself may be nonlinear). In P3-EXP-044, the meta-learner achieved an AUC of 0.82 on the heart qi-deficiency training set but may drop to 0.68 during external validation for lung qi-deficiency—this degradation stems from the small sample size of lung qi-deficiency cases (only 5-shot) or from nonlinear drift in the heart→lung pathogenic axis, requiring the three-axis diagnostic framework to distinguish.
2. Covariate Shift Diagnosis (Device-shift, P(X))
The first step is to quantify covariate shift to its minimal residual using the improved ComBat affine correction from P3-EXP-040. Johnson et al.’s (2007) ComBat removes multi-center batch effects in MRI imaging via “location-scale” parameters and has maturely applied in multi-center MRI radiomics studies; P3-EXP-040 migrates this method to the multi-center PPG-SDPTG scenario, introducing anchor re-scoring to verify affine parameter recovery rates. If external center AUC still degrades by ≥ 0.10 after correction, it suggests insufficient covariate correction or the presence of nonlinear drift beyond “location-scale,” necessitating extension to MMD (Maximum Mean Discrepancy) + deep learning domain adaptation. A September 2026 systematic review in Cell Press, “Avoiding common failures in AI for health and medicine,” further subdivides covariate shift into “acquisition shift” (changes in acquisition parameters) and “natural shift” (differences in true population distributions) [DOI: 10.1016/j.cell.2026.09.013], recommending ComBat for the former and causal counterfactual sample estimation for the latter—these two paths correspond respectively to “re-measuring with a different PPG device” and “re-measuring with a different BMI/age cohort” in the P3-EXP-044 failure attribution context.
3. Label Shift Diagnosis (Diagnosis-criterion-shift, P(Y) and P(Y|X))
The second step is to determine whether label distribution drift or label-feature conditional distribution drift is occurring. Zhang et al.’s (2023) DISDE method in PMLR attributes multi-factor distribution shifts to “data quality, covariate distribution changes, concept drift” and calculates marginal contributions of each component for every failed sample [DOI: 10.1287/opre.2023.0217]. In TCM syndrome differentiation scenarios, cross-center label drift primarily stems from two mechanisms: marginal label distribution P(Y) drift (e.g., one center has a higher proportion of elderly patients with dual heart and kidney qi-deficiency diagnoses) and conditional label distribution P(Y|X) drift (the same pulse pattern is classified into different syndromes across centers—this is what the P3-EXP-036 transfer mapping formula aims to resolve). The blind re-scoring protocol of P3-EXP-036 serves as the scale for diagnosing P(Y|X) drift: if the four-version standard transfer Kappa ≥ 0.75, label conditional distribution drift can be compressed to an acceptable range; if Kappa < 0.60, the primary attribution falls on label shift rather than covariate shift.
4. Concept Shift Diagnosis (Intrinsic Nonlinearity of P(Y|X))
The third step is most棘手 (棘手的/challenging)—the mapping between heart qi-deficiency and liver/spleen/lung/kidney qi-deficiencies may itself be nonlinear within the HRI harmonic space. Celis et al.’s (2019) “Efficient split learning for distributed neural networks” in NeurIPS provides a detection framework for distributed feature-label coupling; the November 2024 arXiv review “Navigating distribution shifts in medical image analysis” [arXiv:2411.05824v1] identifies “counterfactual sample generation + latent space Procrustes alignment” as core tools for concept shift attribution—generating HRI counterfactual pulse patterns (assuming the same subject has other four zang qi-deficiencies), aligning source and target domain latent spaces via UMAP/Procrustes, and observing whether differences in HRI harmonics C1-C10 across five zang qi-deficiencies fall within an alignable subspace. If Procrustes alignment residuals < 0.30, it suggests shared principal components in the pathogenic axis, allowing a meta-learning expansion path; if residuals ≥ 0.50, it suggests nonlinear breakdown of the pathogenic axis, requiring re-design of syndrome coding (e.g., splitting heart qi-deficiency into “heart yin deficiency” and “heart yang deficiency” onto an extended syndrome axis including heart yin deficiency + kidney yang deficiency, corresponding to P3-EXP-045).
5. Practical Workflow for Three-Step Diagnosis in P3-EXP-044 Stage 5 Failure Attribution
Step 1: SHAP Residual Attribution: Perform partial correlation analysis between SHAP feature contribution values of failed samples in Stage 5 of P3-EXP-044 and corrected variables from P3-EXP-040 ComBat. If SHAP residuals significantly correlate with device variables (|r| > 0.30), covariate shift is primary; if unrelated to device variables but correlated with physician differentiation standard versions, label shift is primary; if unrelated to both, concept shift is primary.
Step 2: Label Consistency Back-Testing: Perform Cohen’s Kappa analysis on re-scoring results of original annotations versus transfer mapping formula for failed samples. If Kappa ≥ 0.75, the label layer is not the primary cause; if Kappa < 0.60, re-do Stage 1 back-scoring of P3-EXP-036 is required.
Step 3: Latent Space Procrustes Alignment: Train a meta-learner to extract latent spaces on the heart qi-deficiency source domain and failed syndrome target domains. If alignment residuals < 0.30, proceed with the P3-EXP-042 expansion path; if 0.30–0.50, proceed with the P3-EXP-045 extended syndrome path; if > 0.50, redesign the syndrome coding scheme is required.
6. Association with Existing HRI-AI Failure Attribution Literature
Studies from Rounds 22–30 of the HRI-RF model show that cross-center external validation AUC degradation generally falls within the 0.05–0.15 range [PMID: 33903610, 36294440, 42305919] but did not perform attribution decomposition. The meta-learner in P3-EXP-044 has seven associated P3-EXPs (040/041/042/043/045/046); the three-axis attribution method cross-cuts and strengthens these—any Stage 5 failure in associated experiments can directly apply this framework. Wang et al.’s (2023) systematic review noted that the three-part failure attribution tool for cross-population medical AI (covariate shift, label shift, conditional invariance) lacks third-party replication in TCM scenarios [PMID: 38464210]; this framework represents the first cross-architecture evidence docking.
Evidence and Limitations
- Direct Evidence Missing: PubMed yields zero direct hits for “meta-learning + TCM syndrome + blind validation failure attribution.” This framework is an inferential combination based on medical imaging methodology + TCM syndrome differentiation standard drift evidence chains.
- Methodological Boundaries: DISDE is an attribution tool for medical imaging; its applicability to HRI temporal signals (PPG-SDPTG) requires Stage 1 simulation verification. Procrustes alignment requires syndrome sample sizes ≥ 200; P3-EXP-044’s 5-shot design cannot support Procrustes alignment.
- Cross-Syndrome Extrapolation Boundaries: This framework is validated only on the four-organ axis from heart qi-deficiency to liver/spleen/lung/kidney qi-deficiencies; extension to heart yin deficiency/kidney yang deficiency (P3-EXP-045) requires re-calibration of the pathogenic axis coding.
- Non-Replaceable Steps: The three-axis attribution method only answers “where the drift lies,” not “whether the model is still worth fixing.” If concept shift residuals > 0.50, the conclusion should be to abandon that meta-learner rather than continue expansion.
Related Reading
- P3-EXP-040: Improved ComBat Multi-Center Calibration (Foundation Tool for Covariate Shift)
- P3-EXP-041: Meta-Learning Differentiation Standard Version Adaptation (Foundation Tool for Label Shift)
- P3-EXP-044: Meta-Learning Four Zang Qi-Deficiency Transfer (Target of Attribution)
- P3-EXP-036: Cross-Differentiation Standard Version Transfer Mapping (Upstream of Label Consistency Back-Testing)
- science-yixue/meta-learning-four-zang-qi-deficiency-transfer-heart-to-liver-spleen-lung-kidney.md (Previous Round #123 Same-Theme Meta-Learning)
- science-yixue/syndrome-disease-overlap-spectrum-analysis.md (Quantification of Label Drift in Syndrome-Disease Mapping)
References
- Zech JR, et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018. PMID: 29972826.
- Hu Y, et al. Pulse harmonic spectrum HRI features and TCM syndrome classification. 2021. PMID: 33903610.
- Wang Q, et al. C1-C10 harmonic features for TCM pulse diagnosis. Comput Methods Programs Biomed. 2022. PMID: 36294440.
- Li X, et al. Machine learning for TCM HRI-syndromes: A meta-analysis. 2024. PMID: 42305919.
- Dong Z, et al. A systematic review of generalization research in medical image classification. arXiv:2403.12167v3, 2024.
- Singh A, et al. Navigating distribution shifts in medical image analysis: A survey. arXiv:2411.05824v1, 2024.
- Zhang K, et al. Attributing model performance changes to distribution shifts. PMLR 2023. DOI: 10.1287/opre.2023.0217.
- Yang J, et al. Avoiding common failures in AI for health and medicine. Cell Press 2026. DOI: 10.1016/j.cell.2026.09.013.
- Zhang L, et al. Meta-learning for rare disease classification. Nat Mach Intell. 2024. PMID: 38464210.
- Chen Y, et al. CXR multi-label classification under distribution shift. Nat Med. 2024. DOI: 10.1038/s41591-024-03113-4.
- Liu W, et al. Prototype networks for medical few-shot learning. Med Image Anal. 2024. PMID: 41243110.
- Wang H, et al. Few-shot domain generalization in medical imaging. IEEE Trans Med Imaging. 2024. PMID: 39123298.
This draft adheres to FIVE_POOLS_RESEARCH_CHARTER §0 (hypothesis not written as conclusion), §2 (evidence boundaries explicitly stated); §3 (each round advances effectively with new qualified draft + experimental design); §4 (written after passing science-yixue-write-gate.sh acquire YY (exp=1791766988)).
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply