三质分型信度要有多高才算够用?——一张把及格线变成数字的查询表

Written by

in

Language

Evidence level: B

分型工具的信度不是”越高越好”的学术追求,而是一条可以反推的及格线:你想检测多大的真实效应,就至少要有多高的 kappa——数字不够,样本量来凑,但放大是有上限的。

正文

“先分型、再对型下药”是三质(瓦塔/皮塔/卡法)、三因、中医体质的共同逻辑。但上一篇文章(#351)已经算过一笔账:分组标签的信度不足,会像透镜上的污点一样,把真实的组间差异系统性稀释。那么问题自然来了——信度要有多高才算”够用”? 这篇文章给出一个可操作的答案:把”及格线”反推成一张查询表,让你在设计实验的第一天就知道,你的分型工具到底达不达标。

及格线从哪来:d≥0.3 的设定逻辑

任何”及格线”都需要一个锚点。本文采用的锚点是观测效应量 d≥0.3——这是心理学与行为科学公认的”小到中等”效应边界:低于它,组间差异肉眼不可辨,也容易淹没在个体差异的噪声里。换句话说,一个三质研究如果连 d=0.3 的效应都测不出来,那它宣称的”某体质对某通路更有效”就没有统计上的立足之地。

把锚点定在观测端而非真实端,是因为观测值才是你最终拿到的数字。真实效应可能很大,但经过信度稀释后,落在你报表上的可能已经小得可怜。

反推公式:从及格线算出最小 kappa

上一篇文章引用的 2025 年”可靠性悖论”推导给出了一条干净的衰减公式:观测效应量 ≈ 真实效应量 × sin(π/2·κ)(PMID 41169915)。把这条公式倒过来用,就得到”最小可接受信度”:

kappa ≥ (2/π) × arcsin(0.3 / d真实)

代入几组典型值,结果令人警醒:若真实效应 d=0.5(中等效应),你需要 kappa≥0.41 才能保证观测值不低于 0.3;若真实效应只有 d=0.4(小效应),门槛直接升到 kappa≥0.54;反过来,只有当你赌真实效应高达 d=1.0 时,kappa≥0.19 才勉强够用。

查询表:把数字摆上桌面

把 kappa 从 0.2 扫到 0.8,配合经典结果”二分类错分的渐近相对效率恰好等于 kappa”(PMID 7569489),可以合成一张实用查询表:

分型信度 κ 观测/真实效应比(sin(π/2·κ)) 样本量放大系数(1/κ)
0.20 0.31 5.0
0.30 0.45 3.3
0.40 0.59 2.5
0.50 0.71 2.0
0.60 0.81 1.7
0.70 0.89 1.4
0.80 0.95 1.2

这张表回答两类问题:第一,我现有的分型工具 kappa=0.35,那它最多只能”看见”真实效应的 52%——想测 d=0.5 的效应,观测值只有 0.26,不及格;第二,如果我只想用 kappa=0.4 的工具,那样本量必须按 1/0.4=2.5 倍放大,才能找回丢失的功效。注意放大系数在低信度区上升极快:kappa 从 0.4 降到 0.2,放大系数从 2.5 暴涨到 5.0——与其多招两倍被试,不如先把分型工具的信度提上去,这是查询表最重要的决策信号。

三质研究的现实坐标:你在表的哪一格?

把查询表套到三质研究的实际数据上,坐标非常清晰。2013 年 15 名阿育吠陀医生对 20 名受试者的评估研究显示,体质(prakriti)评估的平均加权 kappa 只有 0.28(PMID 24191170)——按表查询,这只能”看见”真实效应的 43%,意味着靠医生临床评估分组的研究,只能检测到 d≥0.7 的粗大效应,中小效应全部沉底。2012 年另一项脉诊与体质诊断重复性研究同样报告了医生评估的低可重复性(PMID 27257530)。

但同一张表也给出了希望:2022 年一项开发三质自评问卷的研究报告,76 名成人的重测 ICC 三个质全部超过 0.8(PMID 34548242)——按表查询,kappa≈0.8 可以”看见”真实效应的 95%,放大系数仅 1.2。结构化问卷/仪器评估的路径,把三质研究从”只能测大效应”推进到了”能测中效应”的区间。这也解释了为什么现代三质研究几乎清一色改用问卷而非纯临床评估。

功效-信度联合曲面:别只算一条线

查询表是静态的,但实验设计是动态的——kappa 和样本量不是二选一,而是联合决定的。更完整的做法是扫一张功效-信度联合曲面:kappa 取 0.2-0.8 的梯度(对应工具从”差”到”优秀”),样本量取 30-300 的梯度(覆盖小样本探索到大样本确证),对每个格子计算”在此信度与样本量下,检测到 d≥0.3 观测效应的统计功效”。曲面会告诉你:同样的 100 人样本,kappa=0.3 时功效可能只有 30%,kappa=0.7 时却能到 90%——信度差的工具,样本量翻倍也追不回来。

这里还要补一个常被忽视的点:kappa 本身是个估计值,需要足够的样本才能估得准。2005 年一篇信度研究的方法学综述指出,kappa 的置信区间宽度随样本量收窄,低于 30 对评估时区间宽到几乎无信息(PMID 15733050)。信度标定实验本身也要做功效分析——如果你的信度研究只有 20 人,kappa 的 95% 置信区间可能横跨”差”到”好”,后续所有校正都是空中楼阁。

与易学体系的呼应

“及格线反推”的方法论,本质上是易学”取用神”的统计学版本:先定你要达成什么(观测效应 ≥ d),再反推你需要什么(最小信度 kappa),最后决定你付出什么(样本量放大)。古人断命理讲究”用神得力”——工具不得力,结论就虚;现代研究讲究”信度达标”——分型不达标,疗效就稀释。二者殊途同归:都在回答同一个问题——你的工具,配得上你的结论吗?

证据与局限

已验证: 二分类错分时渐近相对效率 ARE=kappa,样本量按 1/kappa 放大(PMID 7569489);sin(π/2·κ) 衰减公式有 2025 年方法学推导支撑(PMID 41169915);三质医生评估 kappa≈0.28(PMID 24191170)与低可重复性(PMID 27257530);自评问卷重测 ICC>0.8(PMID 34548242);kappa 估计精度随样本量变化的规律(PMID 15733050)。

部分验证: 衰减公式在连续潜变量+分类噪声场景有推导,但三质三分类场景的曲线形状尚未被真实数据直接验证(待 #658 裁决);”d≥0.3 及格线”沿用行为科学惯例,尚未在三质临床场景标定最小重要差异。

待验证: d=0.3 作为三质研究及格线的合理性(需基于真实三质研究效应量分布标定);功效-信度联合曲面在不同问卷/仪器下的实测复现;kappa 估计置信区间收敛所需的最小样本量在三质场景的验证;低信度工具”样本量翻倍也追不回”的量化边界。

相关阅读

证据等级: B(ARE=kappa、衰减公式、三质信度现状均有 PubMed 直接文献支撑;”最小可接受阈值查询表”本身属方法学数值推演与设计建议,无直接研究)
参考文献: PMID 7569489, PMID 41169915, PMID 24191170, PMID 27257530, PMID 34548242, PMID 15733050


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

How High Should the Reliability of Three-Factor Classification Be? A Lookup Table That Turns the Passing Threshold into a Number

Evidence level: B

The reliability of typing tools is not an academic pursuit of “the higher, the better,” but a reversible passing threshold: To detect a specific magnitude of true effect, you must achieve at least that level of kappa. If the number is insufficient, sample size can compensate, but there is an upper limit to amplification.

Main Text

The logic shared by Tridosha (Vata/Pitta/Kapha), Three Factors, and Traditional Chinese Medicine constitutions is “classify first, then treat based on classification.” However, as calculated in the previous article (#351), insufficient reliability of grouping labels acts like a smudge on a lens, systematically diluting true between-group differences. The question naturally arises: How high must reliability be to be “sufficient”? This article provides an actionable answer: reverse-engineer the passing threshold into a query table so you know on day one of your experimental design whether your typing tool meets the standard.

Where the Passing Threshold Comes From: The Logic of Setting d ≥ 0.3

Any passing threshold requires an anchor. This article adopts observed effect size d ≥ 0.3 as the anchor—a widely recognized boundary in psychology and behavioral science for “small to medium” effects. Below this threshold, between-group differences are visually indistinguishable and easily drowned out by individual difference noise. In other words, if a Tridosha study fails to detect even a d=0.3 effect, its claim that “a certain constitution is more effective for a certain pathway” lacks statistical footing.

Setting the anchor at the observational end rather than the true end is because observed values are the numbers you ultimately receive. The true effect may be large, but after reliability dilution, what falls on your report may be negligibly small.

Reverse-Engineering Formula: Calculating Minimum Acceptable Kappa from the Passing Threshold

The 2025 “Reliability Paradox” derivation cited in the previous article provides a clean attenuation formula: Observed Effect Size ≈ True Effect Size × sin(π/2·κ) (PMID 41169915). Using this formula in reverse yields the “minimum acceptable reliability”:

kappa ≥ (2/π) × arcsin(0.3 / d_true)

Substituting typical values yields startling results: If the true effect is d=0.5 (medium), you need kappa ≥ 0.41 to ensure the observed value does not drop below 0.3; if the true effect is only d=0.4 (small), the threshold rises directly to kappa ≥ 0.54; conversely, you only barely suffice if you gamble on a true effect as high as d=1.0, requiring kappa ≥ 0.19.

Query Table: Placing Numbers on the Desk

Scanning kappa from 0.2 to 0.8, combined with the classic result that “the asymptotic relative efficiency of misclassification in binary classification equals kappa exactly” (PMID 7569489), synthesizes a practical query table:

Typing Reliability κ Observed/True Effect Ratio (sin(π/2·κ)) Sample Size Amplification Factor (1/κ)
0.20 0.31 5.0
0.30 0.45 3.3
0.40 0.59 2.5
0.50 0.71 2.0
0.60 0.81 1.7
0.70 0.89 1.4
0.80 0.95 1.2

This table answers two questions: First, if my existing typing tool has kappa=0.35, it can only “see” 52% of the true effect—trying to detect a d=0.5 effect would yield an observed value of only 0.26, which is insufficient; Second, if I only wish to use a tool with kappa=0.4, the sample size must be amplified by 1/0.4 = 2.5 times to recover lost power. Note that the amplification factor rises extremely rapidly in low-reliability zones: as kappa drops from 0.4 to 0.2, the factor explodes from 2.5 to 5.0—it is better to improve the reliability of the typing tool than to recruit twice as many subjects, which is the most critical decision signal from this query table.

Real-World Coordinates for Tridosha Studies: Where Do You Fall on the Table?

Applying the query table to actual Tridosha study data yields very clear coordinates. A 2013 study showing 15 Ayurvedic physicians assessing 20 subjects reported an average weighted kappa for constitution (prakriti) assessment of only 0.28 (PMID 24191170)—according to the table, this can only “see” 43% of the true effect, meaning studies relying on physician clinical assessment for grouping can only detect coarse large effects of d ≥ 0.7, with all small and medium effects sinking to the bottom. Another 2012 study on the repeatability of pulse diagnosis and constitution diagnosis also reported low repeatability in physician assessments (PMID 27257530).

However, the same table also offers hope: A 2022 study reporting on the development of a self-assessment questionnaire for Tridosha found that in a cohort of 76 adults, the three reliability coefficients (ICC) for all three dimensions exceeded 0.8 (PMID 34548242)—according to the table, kappa ≈ 0.8 can “see” 95% of the true effect, with an amplification factor of only 1.2. The path of structured questionnaires/instrumented assessment advances Tridosha research from “only able to measure large effects” to the interval where “medium effects can be measured.” This explains why modern Tridosha research almost exclusively switches to questionnaires rather than pure clinical assessment.

Power-Reliability Joint Surface: Do Not Calculate Only One Line

The query table is static, but experimental design is dynamic—kappa and sample size are not mutually exclusive but jointly determined. A more complete approach is to scan a power-reliability joint surface: taking kappa at gradients from 0.2 to 0.8 (corresponding to tools ranging from “poor” to “excellent”), and sample size at gradients from 30 to 300 (covering small-sample exploration to large-sample confirmation), calculating the “statistical power to detect an observed effect of d ≥ 0.3 under each combination of reliability and sample size.” The surface will tell you: with the same 100 subjects, power might be only 30% when kappa=0.3, but could reach 90% when kappa=0.7—a tool with poor reliability cannot be caught up to by doubling the sample size.

Here is also a point often overlooked: kappa itself is an estimate requiring sufficient samples to be estimated accurately. A 2005 methodological review on reliability studies pointed out that the width of kappa’s confidence interval narrows with sample size, becoming so wide at below 30 subjects that it provides almost no information (PMID 15733050). The reliability calibration experiment itself also requires power analysis—if your reliability study has only 20 subjects, kappa’s 95% confidence interval might span from “poor” to “good,” rendering all subsequent corrections on shaky ground.

Resonance with Yijing Studies Systems

The methodology of “reverse-engineering the passing threshold” is essentially the statistical version of Yijing Studies’ “using the useful god” (taking the deity): First determine what you aim to achieve (observed effect ≥ d), then reverse-engineer what you need (minimum reliability kappa), and finally decide what you must pay for (sample size amplification). Ancient fortune-telling emphasizes “the useful god must be effective”—if the tool is ineffective, the conclusion is hollow; modern research emphasizes “meeting reliability standards”—if typing is non-compliant, efficacy is diluted. Both converge on the same point: Does your tool deserve your conclusion?

Evidence and Limitations

Verified: The asymptotic relative efficiency ARE=kappa for binary misclassification, sample size amplification by 1/kappa (PMID 7569489); the sin(π/2·κ) attenuation formula is supported by 2025 methodological derivation (PMID 41169915); Tridosha physician assessment kappa ≈ 0.28 (PMID 24191170) and low repeatability (PMID 27257530); self-assessment questionnaire retest ICC > 0.8 (PMID 34548242); the规律 of kappa estimation precision varying with sample size (PMID 15733050).

Partially Verified: The attenuation formula has derivation in continuous latent variable + classification noise scenarios, but the curve shape in the Tridosha three-class scenario has not been directly verified by real data yet (pending adjudication in #658); the “d ≥ 0.3 passing threshold” follows behavioral science conventions and has not yet been calibrated for the minimum important difference in Tridosha clinical scenarios.

To Be Verified: The rationality of d=0.3 as a passing threshold for Tridosha studies (requires calibration based on actual effect size distributions in real Tridosha studies); empirical replication of the power-reliability joint surface across different questionnaires/instruments; verification of the minimum sample size required for kappa confidence interval convergence in Tridosha scenarios; quantification of the boundary where “doubling sample size fails to recover” for low-reliability tools.

Related Reading

Evidence Level: B (ARE=kappa, attenuation formula, and current Tridosha reliability status all have direct PubMed literature support; the “minimum acceptable threshold query table” itself belongs to methodological numerical deduction and design suggestions, with no direct study)
References: PMID 7569489, PMID 41169915, PMID 24191170, PMID 27257530, PMID 34548242, PMID 15733050


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *