Evidence level: B
判读者之间的相关系数偏低,可能是「几个人尺子不一样」,也可能是「同一个人自己的尺子在漂」——不加重测轮,这两种原因永远混在一个数里分不开。
正文
前一篇判读者间相关的实测分布研究给多中心设计定下了规格:多位判读者独立判读同批样本,用判读者间相关 ρ 来量化「分歧有多大」。但那个设计有一个天然的盲区——它只分解了判读者之间的分量。如果 ρ 偏低,原因可能有两种:第一种,几位判读者的尺子本来就不一样,你严我宽,彼此错位;第二种,每一位判读者自己的尺子在时间上就不稳,今天判的和两周后判的,同一个样本给出不同的读数。这两种原因在只测一轮的设计里完全混叠,看到的低 ρ 既可能是「人不齐」,也可能是「人自己在漂」。
把两者拆开的办法在测量学里是现成的:让至少三位判读者对同批至少三十个样本隔两周重读一遍,给每位判读者建立「判读者内重测一致性」(ICC intra),与原有的判读者间相关(ρ inter)并列比较。数学上这走的是概括理论的多层方差分解路线:把总方差拆成样本间、判读者间、判读者内时间漂移三层,各层贡献用混合模型同时估计(PMID 33818118)。这正是前一篇设计缺的那一层——加一个「时间」维度,混合的两种原因就有了各自的出口。
这个拆分值得做吗?值得。因为「同一个人隔段时间判同一个东西会漂」在生理测量里是实测过的常态:健康成人心率变异性指标一周内重测,时间域指标一致性尚可、频域指标明显更差,静息态信度系数只在 0.50 到 0.83 之间(PMID 39854840)。也就是说,即便是训练有素的标准化测量,「同一把尺子隔一周不复原」并不是罕见事故,而是频域类读数的默认属性。判读构型这种主观成分更重的任务,没有理由假设自己比仪器读数更稳。而重测研究还有一个常被忽略的前置问题:重测信度的估计本身对样本量极其敏感,样本不足时重测系数的估计会系统性失真(PMID 40119099)——所以重测轮不是「加两个时点」那么简单,30 例 3 人的规格是按估计精度反推的下限,不是拍脑袋。
还有一个更隐蔽的坑必须点名:普通的信度系数对「整体往上或往下漂」这类系统性偏置几乎不敏感。方法学文献早已指出,在存在系统偏置时,相关系数、配对检验与组内相关系数会给出互相矛盾的结论,而绝对一致性定义的组内相关系数是三者中唯一能反映偏置的,但仍须配均值检验确认(PMID 11091941);组内相关系数的绝对一致版与一致性版近乎相等时,才能宣布「系统偏置可忽略」(PMID 31329615)。落到本文的设计上就是一条硬规矩:判读者内重测一致性必须报绝对一致口径,只报一致性口径的重测轮等于白测——漂移恰好整体同向时它照样给你高分。
裁决标准三分支,事先写死:若判读者内重测一致性与判读者间相关之差小于 0.1,说明时间漂移可忽略,前一篇的多中心设计不必加重测轮,省一轮成本;若两者之差在 0.1 到 0.2 之间,判读内漂移与判读者间分歧同量级,任何 ρ 的报告必须同时报两个分量、不得只报合并值;若差达到 0.2 以上,则时间漂移是主导项——任何不加重测校正的 ρ 估计都作废,单人判读的长期可用性存疑,判读规程必须改成「定期重校准」模式。
这一层分解对易学研究为什么关键?因为整个「五脏辨证标签能不能当分组变量」的前提问题,正卡在标签信度上:开放人群里中医医师间诊断一致性被报告为低、作者称为令人担忧(PMID 30383398),但另一侧证据说训练可以改善、量表化可行。冲突双方都成立的可能性完全存在——因为「低一致性」这个观察本身可能就有一部分来自判读者内的时间漂移,而非判读者之间真实的理论分歧。不拆开,这场关于「辨证标签可不可靠」的争论就会永远在两个不同误差源的混淆里打转。
证据与局限
已验证:多层方差分解可同时估计判读者间与判读者内分量(概括理论框架成熟);频域生理读数一周重测信度 0.50 至 0.83、差于时间域,为「时间漂移量级真实存在」提供直接实测锚点;绝对一致性口径的组内相关系数是察觉系统偏置的必要口径。
部分验证:重测信度估计的样本量敏感性有直接方法学文献,但「3 人 30 例」的具体规格是按本项目判读构型任务反推的,未在气功外气构型判读上实测过。
待验证:三分支判据(差小于 0.1 免重测轮、达到 0.2 强制校正)为项目内部仿真推演,需按本设计采集真实数据后回填;判读者内漂移与判读者间分歧在真实辨证任务中的实际占比未知。
相关阅读
- 判读者间相关 ρ 的实测经验分布:分档表三档里现实世界最常落在哪一档
- 判读者的尺子到底会漂多少:临界值与真人实测差了两三倍
- 求稳是记忆的下游还是独立现象:三臂设计裁决 ICC 抵消公式
- 池1 · 共振意识场
证据等级: B
参考文献: PMID 40119099, PMID 33818118, PMID 39854840, PMID 30383398, PMID 31329615, PMID 11091941
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
L1 网络状态:本主题对应的「判读者内重测 × 判读者间相关联合分解」边尚未进入 L1 对应网络,已登记 science-yixue-l1-gaps.md。最近邻边为 E-060 标签前提(五脏辨证作分组标签 → 标签信度,状态 冲突(未裁决),原文摘录:「中医医师间诊断一致性在开放人群中『低(low)』,作者称为『令人担忧』(PMID 30383398);同系列第二篇指出偶然校正一致性统计常被误用、建议 Gwet AC2(PMID 31730403 / 31634000 / 31730404);但训练可改善(PMID 19538100)、量表化可行」)——本文已按规则把冲突双方(标签一致性低 vs 训练可改善)并列写出,未只挑一边。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Joint decomposition of within-reader and between-reader correlation: how much does biweekly re-reading extract from ρ to isolate within-reader drift
Evidence level: B
Joint Decomposition of Intra-Reader and Inter-Reader Correlation: How Much Can Repeated Reading Every Other Week Extract from ρ
Intra-Reader Drift: A systematic shift in an individual’s measurement scale over time, often due to subtle changes in perception or environmental context.
Inter-Reader Correlation: The degree of agreement between different individuals using the same scale, reflecting consistency across observers.
The low correlation coefficient among readers may stem from either “different rulers” (inter-Reader variation) or “a reader’s own ruler drifting” (intra-Reader drift). Without a repeated measurement cycle, these two sources of variation remain inseparable within the single correlation value.
Body
The preceding study on inter-rater distribution established specifications for a multi-center design: multiple independent raters assess the same batch of samples, using inter-rater correlation ρ to quantify “how much disagreement exists.” However, that design has an inherent blind spot—it only decomposes variance between raters. If ρ is low, the cause could be either of two factors: first, the raters’ scales differ fundamentally (one strict, one lenient), leading to misalignment; second, each rater’s own scale is unstable over time, yielding different readings for the same sample assessed today versus two weeks later. In a single-round design, these causes are completely confounded; the observed low ρ could reflect either “raters not aligned” or “raters drifting themselves.”
The method to disentangle these two factors is standard in measurement science: have at least three raters re-assess at least thirty samples after a two-week interval, establishing “intra-rater reliability” (ICC intra) alongside the existing inter-rater correlation (ρ inter). Mathematically, this follows the multilevel variance decomposition route of Generalizability Theory: splitting total variance into three layers—between samples, between raters, and intra-rater temporal drift—with each layer’s contribution estimated simultaneously via mixed models (PMID 33818118). This is precisely the missing layer in the preceding design—adding a “time” dimension gives each of the two mixed causes its own exit.
Is this decomposition worth undertaking? Yes, because “the same person drifting when re-assessing the same thing after some time” is empirically established as normal in physiological measurement: heart rate variability indices in healthy adults re-assessed within one week show acceptable time-domain consistency but significantly worse frequency-domain reliability, with resting-state reliability coefficients ranging only between 0.50 and 0.83 (PMID 39854840). Thus, even with trained standardized measurement, “the same scale not reproducing itself after one week” is not a rare accident but the default attribute of frequency-domain-like readings. Since Yijing Studies involves tasks with heavier subjective components, there is no reason to assume they are more stable than instrument readings. Moreover, re-test studies have an often-overlooked prerequisite issue: the estimation of re-test reliability itself is extremely sensitive to sample size; insufficient samples lead to systematic distortion in the re-test coefficient estimate (PMID 40119099)—so the re-test round is not simply “adding two time points”; the specification of 30 cases and 3 raters is the lower limit derived from required estimation precision, not arbitrary.
There is another hidden pitfall that must be named: ordinary reliability coefficients are nearly insensitive to “systematic upward or downward drift.” Methodological literature has long pointed out that, under systematic bias, correlation coefficients, paired tests, and intraclass correlation coefficients (ICC) yield contradictory conclusions, while the ICC defined by absolute consistency is the only one among them that reflects bias—but it still requires mean tests for confirmation (PMID 11091941); only when the absolute-consistency ICC and the consistency-based ICC are nearly equal can we declare “systematic bias negligible” (PMID 31329615). Applied to this design, this becomes a hard rule: intra-rater re-test consistency must be reported in absolute-consistency terms; reporting only the consistency-based ICC renders the re-test round pointless—even if drift occurs entirely in the same direction, it still yields high scores.
The decision criteria have three branches, fixed in advance: if the difference between intra-rater re-test consistency and inter-rater correlation is less than 0.1, temporal drift is negligible, so the preceding multi-center design need not add a re-test round, saving one cycle of cost; if the difference lies between 0.1 and 0.2, intra-rater drift and inter-rater disagreement are of comparable magnitude, so any ρ report must simultaneously provide both components and not just the combined value; if the difference reaches 0.2 or more, temporal drift is the dominant factor—any ρ estimate without re-test correction becomes invalid, long-term usability of single-rater assessment is questionable, and the assessment protocol must shift to a “periodic re-calibration” mode.
Why is this decomposition critical for Yijing Studies? Because the entire premise—”can organ-system differentiation labels serve as grouping variables”—hinges on label reliability: in open populations, diagnostic consistency among TCM physicians has been reported as low, with authors calling it concerning (PMID 30383398), yet evidence on the other side suggests training can improve it and standardized scales are feasible. The possibility that both sides hold true is entirely real—because the observation of “low consistency” itself may partly stem from intra-rater temporal drift rather than genuine theoretical disagreement among raters. Without decomposition, the debate over “whether differentiation labels are reliable” will endlessly circle within confusion of two distinct error sources.
Evidence and Limitations
Verified: Multi-level variance decomposition can simultaneously estimate inter-rater and intra-rater components (confirming the maturity of the theoretical framework); frequency-domain physiological readings show reliability coefficients of 0.50 to 0.83 upon retesting over one week, which is inferior to time-domain metrics and provides direct empirical anchors for “the reality of temporal drift magnitudes”; the intraclass correlation coefficient based on absolute consistency serves as the necessary metric for detecting systematic bias.
Partially Verified: While literature exists on methodological sensitivity regarding sample size for reliability estimation, the specific specification of “3 raters and 30 cases” was reverse-engineered from the task requirements of this project’s reading configuration and has not yet been empirically tested on Qimen Dunjia (a traditional Chinese metaphysical system for timing and spatial pattern analysis) external energy configuration readings.
Pending Verification: The three-branch criteria (re-testing skipped if difference < 0.1; mandatory correction if ≥ 0.2) are currently internal simulations and require real data collection according to this design before back-filling; the actual proportion of intra-rater drift versus inter-rater disagreement in real diagnostic tasks remains unknown.
Related Reading
- Empirical distribution of inter-rater correlation ρ for Yijing Studies readers: stratification into three tiers, showing where reality most frequently falls
- How much does the ruler for assessing Yijing Studies readers drift? Critical thresholds and observed differences of two to three times in real-world testing
- Is stability a downstream effect of memory or an independent phenomenon? The three-arm design adjudicates the ICC cancellation formula
- Pool 1 · Resonance Consciousness Field
Evidence level: B
References: PMID 40119099, PMID 33818118, PMID 39854840, PMID 30383398, PMID 31329615, PMID 11091941
Evidence level: A = direct PubMed support; B = scientific analogy or indirect evidence; C = traditional consensus or preliminary exploration.
L1 network status: The combined decomposition of “intra-rater retest × inter-rater correlation” for this topic has not yet entered the L1 corresponding network; it has been registered in science-yixue-l1-gaps.md. The nearest neighbor edge is the E-060 label premise (organ differentiation as grouping labels → label reliability, status conflict (unresolved)), with the original excerpt: “Consistency of diagnosis among TCM physicians in an open population is ‘low (low)’, described by the authors as ‘concerning’ (PMID 30383398); the second paper in the same series notes that chance-corrected consistency statistics are often misapplied and recommends Gwet AC2 (PMID 31730403 / 31634000 / 31730404); however, training can improve (PMID 19538100), and scaling is feasible.” This article lists both conflicting positions (low label consistency vs. trainable improvement) side-by-side per the rules, rather than selecting only one.
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.