Evidence level: C
两位专家判定不一致,不等于体质标签全是废料——先用 Fleiss κ 把「可学习的标注」和「噪声标注」分开,只让可学习子集进入下游分析,才是对三因体质(prakriti)做实证研究的正确起点。
正文
「三因体质」是阿育吠陀(Ayurveda)的核心分型框架:vata、pitta、kapha 三种 dosha 的组合决定一个人的体质类型。但在实证研究里,它有一个绕不过去的前置问题——两个资深专家独立判定同一个人,结论能对上吗?
已有数据给出的是坏消息与好消息各半。坏消息:Kurande 等人让 15 名执业医师独立检查 20 名受试者,对所有 105 个医师配对计算加权 κ,体质判定的平均 pairwise κ 只有 0.28(fair 到 moderate 之间),脉诊更惨,只有 0.07(PMID 24191170)。另一项嵌套在膝关节骨关节炎 RCT 里的诊断可靠性研究更直接:4 名阿育吠陀专家对 30 名患者做 Fleiss κ 分析,prakriti 和 agni 这类「体质实体」的 κ 大多落在 0 到 0.4 之间,而疾病相关实体的判定反而好得多(PMID 30653338)。也就是说,传统式(凭触诊和经验)的体质判定,一致性显著低于疾病诊断。
好消息来自工具标准化:一项 3416 人的多中心研究里,资深医师与计算机辅助评定工具(Ayusoft)的判定 κ 达到 0.77,一致率 80%(PMID 25336848)。对照 0.28,这个跃升说明低 κ 的主要来源很可能不是「体质不存在」,而是测量方法本身没有统一操作口径——判定依据越靠个人经验,标签噪声越大。
这引出本文的主题:所谓「标签可学习性预筛选」。方法论上,κ 低有三种可能混在一起:真实可学习的类别信号、纯粹的噪声标注、以及类别分布失衡导致的 κ 悖论。近年方法学文献已经警告,Fleiss κ 对评定者数量与类别流行率分布敏感,会得出失真甚至膨胀的估计(PMID 36872574),多评定者一致性系数也需要更稳健的方差推断框架(PMID 38190018)。所以预筛选协议不能只算一个总 κ,而应做四步:
第一,双专家对全部样本独立判定,禁止先商量再判,否则测的是共识而非可靠性。第二,计算 Fleiss κ 加类别间一致性矩阵——不只看总体 κ,还要看混淆集中在哪两个类别之间:如果 vata 与 vata-pitta 之间频繁互判,可能是类别定义重叠(可修复);如果是随机散布的不一致,才是噪声。第三,给每条标注打可学习性评分:两位专家一致的样本进入「可学习」集;不一致样本若落在某个稳定的混淆对里,先做类别定义仲裁后再回收;完全随机的散布样本标为噪声,退出下游。第四,用类别流行率修正后的估计量复核,排除流行率悖论对 κ 的压低或抬升。
对中医体质学(九种体质量表体系)而言,这一预筛选有直接的移植价值:中医体质判定走的是标准化问卷路线,原则上更接近「软件判定 κ=0.77」的形态,但问卷题项与专家判读之间的 κ 分层同样没人系统做过。用同一套「可学习子集」流程先体检再入库,后续任何体质 × 生理指标的关联分析才不会建在噪声标注的地基上。
还有一个常见误用需要提前拦住:不能拿预筛选结果反向宣布「体质构念不存在」。κ 衡量的是测量过程的可靠性,不是构念的实在性——脉诊 κ=0.07 只能说明触诊式判读缺乏可复现的操作定义,不构成对三因学说本体的否定;同样,可学习性筛选淘汰掉的样本,只是「目前这套测量程序下不可靠的标注」,换一套更结构化的采集流程后,其中一部分完全可能被回收。反过来说,可学习子集也不该被神化:两位专家恰好共用同一套训练背景时,一致可能反映的是「同门偏置」而非客观信号,所以理想设计应让两位专家来自不同传承或机构,并在协议里预注册各自的判定依据清单。
落地时的最低报告标准建议为:双判定的 κ 及置信区间、类别间混淆矩阵全表、可学习与噪声子集的样本量与占比、判定依据清单的预注册文本,四项缺一不可。缺少混淆矩阵的 κ 报告,等于只给了一个无法定位病灶的总分;缺少预注册依据清单,则事后无法区分「定义重叠」与「随机噪声」——这两类不一致的修复路径完全不同,混在一起谈只会让后续研究者重复踩坑。
证据与局限
已验证:κ 作为多评定者一致性度量的数学性质与已知失真模式(PMID 36872574、38190018);传统式 prakriti 判定低一致性(κ 0.28 与 0–0.4)是实测事实(PMID 24191170、30653338)。
部分验证:标准化工具能把体质判定一致性拉到 0.77,提示低 κ 可归因于测量口径而非构念本身(PMID 25336848,单工具单中心条件)。
待验证:①「标签可学习性评分」预筛选是否真的能提升下游体质 × 生理指标关联分析的效应量稳定性;②可学习子集筛选在中医九体质量表上是否可复制;③类别间混淆矩阵能否稳定区分「定义重叠」与「随机噪声」两类不一致。
相关阅读
证据等级: C
参考文献: PMID 24191170, PMID 30653338, PMID 25336848, PMID 36872574, PMID 38190018
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
L1 网络状态:该主题尚未进入 L1 对应网络。EDGE_TABLE_science-yixue.md 对「体质/prakriti/阿育吠陀/kappa/κ」零命中;CORRESPONDENCE_NETWORK.md 仅有第 73 行「体质分型 | 量表 | 个体差异的调节变量」的泛提及,无本主题(双专家判定一致性 / 标签可学习性)对应边。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Pre-screening of \”Label Learnability\” for Individuals with Inconsistent Three-Cause Constitution Dual Judgments: Fleiss’ Kappa Distinguishes Signal from Noise
Evidence level: C
Disagreement between two experts does not render constitution labels entirely useless; first applying Fleiss κ to separate “learnable annotations” from “noisy annotations,” and then allowing only the learnable subset to enter downstream analysis, constitutes the correct starting point for empirical research into the three-cause constitution (prakriti).
Body
The “Three-Cause Constitution” (a structural analogy for Ayurvedic body type classification) is the core typological framework of Ayurveda: the combination of the three doshas—vata, pitta, and kapha—determines an individual’s constitution type. However, empirical research reveals an unavoidable preliminary question: Can two senior experts independently assessing the same person reach a consistent conclusion?
Existing data presents mixed results. The negative findings indicate that Kurande et al. had 15 practitioners independently evaluate 20 subjects, calculating weighted κ for all 105 practitioner pairs; the average pairwise κ for constitution determination was only 0.28 (fair to moderate), while pulse diagnosis performed even worse at 0.07 (PMID 24191170). A diagnostic reliability study nested within an RCT for knee osteoarthritis was more direct: 4 Ayurvedic experts assessed 30 patients using Fleiss κ analysis, and κ values for “constitutional entities” such as prakriti and agni mostly fell between 0 and 0.4, whereas disease-related entities were assessed with greater reliability (PMID 30653338). Thus, traditional constitution assessment (relying on palpation and experience) shows significantly lower inter-rater consistency than disease diagnosis.
The positive findings stem from tool standardization: in a multicenter study of 3,416 participants, senior physicians using the computer-aided assessment tool (Ayusoft) achieved a κ of 0.77 and 80% agreement (PMID 25336848). Compared to the 0.28 baseline, this improvement suggests that low κ is likely not due to “non-existent constitution” but rather inconsistent measurement protocols—the more reliance on personal experience, the greater the label noise.
This leads to the paper’s central theme: “learnability pre-screening of labels.” Methodologically, low κ may arise from three confounded sources: genuine learnable class signals, pure noise labeling, and the κ paradox caused by imbalanced class distributions. Recent methodological literature warns that Fleiss κ is sensitive to the number of raters and class prevalence distribution, potentially yielding distorted or inflated estimates (PMID 36872574), and multi-rater consistency coefficients require more robust variance inference frameworks (PMID 38190018). Therefore, the pre-screening protocol cannot rely on a single overall κ but must follow four steps:
First, dual-expert independent assessment of all samples, prohibiting prior consultation before judgment, otherwise the measure reflects consensus rather than reliability. Second, calculate Fleiss κ alongside an inter-class consistency matrix—not merely the overall κ but identifying where confusion concentrates: if vata and vata-pitta frequently swap, this indicates overlapping class definitions (repairable); if inconsistencies are randomly scattered, they represent noise. Third, assign learnability scores to each label: samples agreed upon by both experts enter the “learnable” set; inconsistent samples falling within stable confusion pairs undergo class definition arbitration before reintegration; completely random scatter samples are labeled as noise and excluded from downstream analysis. Fourth, validate using prevalence-corrected estimators to exclude prevalence paradoxes from suppressing or inflating κ.
For Traditional Chinese Medicine constitution studies (the nine-constitution scale system), this pre-screening has direct translational value: TCM constitution assessment follows a standardized questionnaire route, theoretically closer to the “software κ=0.77” model, yet systematic investigation of κ stratification between questionnaire items and expert interpretation remains absent. Using the same “learnable subset” workflow for initial screening before database entry ensures that subsequent constitution × physiological indicator association analyses are not built on noisy labeling foundations.
One common misapplication must be preempted: pre-screening results cannot retroactively declare “constitution as a construct does not exist.” κ measures measurement process reliability, not ontological reality—a pulse diagnosis κ=0.07 merely indicates palpation-based assessment lacks reproducible operational definitions, not that the Three-Cause theory itself is invalid; similarly, samples eliminated by learnability screening are simply “unreliable labels under current measurement protocols,” and some may be recovered with more structured data collection. Conversely, the learnable subset should not be deified: when two experts share identical training backgrounds, their agreement may reflect “same-mentor bias” rather than objective signals; thus, ideal design requires experts from different lineages or institutions, with pre-registered assessment criteria lists in the protocol.
Minimum reporting standards for implementation include: dual-assessment κ with confidence intervals, full inter-class confusion matrices, sample sizes and proportions of learnable vs. noise subsets, and pre-registered assessment criteria lists—all four elements are indispensable. Omitting the confusion matrix reduces κ reporting to an undifferentiated total score, while lacking pre-registered criteria prevents post-hoc distinction between “definition overlap” and “random noise”—these two inconsistency types require fundamentally different correction paths, and mixing them risks repeated pitfalls for future researchers.
Evidence and Limitations
Verified: κ as a mathematical measure of multi-rater agreement aligns with known distortion patterns (PMID 36872574, 38190018); the traditional classification of prakriti yields low consistency (κ 0.28 and 0–0.4), which is an empirical fact (PMID 24191170, 30653338).
Partially Verified: Standardized tools can raise consistency of constitution assessment to 0.77, suggesting that low κ is attributable to measurement scope rather than the construct itself (PMID 25336848, under single-tool, single-center conditions).
Pending Verification: ① Whether a pre-screening based on “label learnability scores” genuinely enhances the stability of effect sizes in downstream analyses linking constitution × physiological indicators; ② whether learnable subsets can be replicated on the TCM Nine-Body Quality Scale; ③ whether confusion matrices between categories can stably distinguish “definition overlap” from “random noise” types of inconsistency.
Related Reading
- IPG Quantification of the Three Doshas Body Types
- Multicenter Reproducibility of the Three Doshas Body Type Frequency Protocol
- Research Pools
Evidence Level: C
References: PMID 24191170, PMID 30653338, PMID 25336848, PMID 36872574, PMID 38190018
Evidence Level: A = direct PubMed support; B = scientific analogy or indirect evidence; C = traditional consensus or preliminary exploration.
L1 Network Status: This topic has not yet entered the L1 corresponding network. EDGE_TABLE_science-yixue.md shows zero hits for “body type/prakriti/Ayurveda/kappa/κ”; CORRESPONDENCE_NETWORK.md only contains a general reference on line 73 regarding “body type classification | scale | moderating variable of individual differences,” without the specific edge corresponding to this topic (dual-expert judgment consistency / label learnability).
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply