Evidence level: C
一个判读者的”不稳”和”不灵”,可能根本是同一个能力的两面——如果是,那套把两者分开诊断的双参数框架就整个作废。
正文
判读三个人看同一批材料时会有两种坏法:一种是「飘」——同一批材料让他隔天重判,结果对不上,噪声大;另一种是「钝」——材料之间真实存在的差异他抓不住,敏感度斜率低,什么都判成”差不多”。在统计模型里,这两件事可以被写成两个参数:噪声方差管”飘”,敏感度斜率管”钝”。前面几篇的分析就把它们当成两个独立注入的机制,各算各的贡献。
但这个”独立”是模型的假设,不是对真人的测量。现实里的直觉恰恰相反:一个判得飘的人,往往也正是因为抓不住材料间的真实差异——「钝」和「飘」在潜在能力上可能是同一个维度的两面。如果真是同源,那么所有把两种不对称分开算的双参数表格就都没有意义,因为根本不存在独立的斜率项——它们必须塌缩成一个单参数的「判读能力」。
现代测量学其实早就给这套分析备好了工具。多面 Rasch 与项目反应理论(IRT)模型中,评分者的”宽严”(severity)与”判别力”(discrimination)本来就是两个参数:有研究用等级反应模型对发散思维评分数据同时估计两位评分者参数,发现当评分者间一致性很高时,宽严与判别力的个体差异对最终分数影响不大(PMID 40558819);而 MRCP(UK) 大型临床考试对 1259 名考官、14 万条配对评分的多面 Rasch 分析,则把考官宽严差从考生能力差里干净地分离了出来——考官宽严只占主效应方差的约 12%(PMID 16919156)。更关键的是一项贝叶斯分层潜特质模型研究:它对大规模作文评分同时估计评分者偏差(宽严)与不可靠性(噪声),发现对最终成绩的影响主要来自不可靠性而非偏差(PMID 29614129)——这个结果在提示噪声的量级更大,但它没有回答这两个参数在人与人之间相不相关。
这正是一块没人填的空白:模型里两参数共存,不等于两参数在个体层面独立。同一批受试者上同时估噪声与灵敏的实操路径已有先例(PMID 30870999);量表形式本身甚至会改变能力维度的结构——有序量表与视觉模拟量表在感知评价上能给出不同结论(PMID 12417797)——所以维度结构必须在指定条件下实测,不能靠跨研究拼。
这个相关有充分的理由可能不为零。第一,两者共享同一个底层能力:抓不住真实差异的人,判读只能靠猜,猜出来的结果既”钝”(斜率低)又”飘”(重判对不上)。第二,两者共享同一个认知资源:注意力和工作记忆差的人,既难以维持稳定的标准,也难以分辨细小的差异。第三,训练会同时改变两者:受过系统训练的判读者通常既更稳也更准。反过来,也可能出现负相关——有些人靠牺牲稳定性换取灵敏(对微小差异过度反应),有些人靠牺牲灵敏换取稳定(把差异都归入中档求稳)。三种情形对应三条完全不同的判读者管理策略,而现在的文献没有任何一条数据能区分它们。
可做的实测很直接:≥30 位判读者 × 同一批 ≥100 例三档材料 × 每人重复判读 3 遍,用多层模型同时估每人的噪声方差与敏感度斜率,再把这两个人际估计拿出来算相关(用后验分布给相关系数一个可信区间)。核心判据三条:若相关绝对值小于 0.3,则两者可分别报告,双参数框架成立;若相关超过 0.6,则真人是单参数的,”钝”与”飘”须合并为一个「判读能力」维度,所有分开算的表格重写为单参数;若相关为负(越稳的人反而越钝),则存在效率与稳定的权衡,判读者筛选须按用途分两套标准,而不是一套能力排序。
对易学侧的启示是:传统语境把”功夫”当作单一维度——火候深了自然又稳又准。本篇把这个单维直觉变成了一个可检验命题:它成立与否,取决于那条人际相关落在三段判据的哪一段。
证据与局限
已验证:评分者宽严与判别力可以作为两个参数从真实大规模数据中分离(PMID 16919156);偏差与不可靠性可同时估计,且后者对成绩的影响量级更大(PMID 29614129)。
部分验证:IRT 框架下双参数估计的收敛性与稳定性在高一致率数据中成立(PMID 40558819);噪声与灵敏可在同一批个体上并行估计(PMID 30870999)。
待验证:噪声方差与敏感度斜率的人际相关——本文全部判据都押在这个从未被直接测过的数字上;量表形式对维度结构的调制(PMID 12417797)也提示结论须绑定具体量表形式。
相关阅读
证据等级: C
参考文献: PMID 29614129, PMID 40558819, PMID 16919156, PMID 30870999, PMID 12417797
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
L1 网络状态:该主题(评分者噪声方差 × 敏感度斜率的人际相关)尚未进入 L1 对应网络——EDGE_TABLE_science-yixue.md 与 CORRESPONDENCE_NETWORK.md 中均无对应边,最近的是 E-060 标签信度 conflict(中医医师间诊断一致性 low,PMID 30383398),属相邻但不同的一条边。故本文证据等级封顶 C,所有结论均属待验证设计,不构成已验证机制。
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Are \”Judged as Drifting\” and \”Judged as Dull\” the Same Thing: Will a Two-Parameter Diagnosis on a Three-Tier Scale Collapse into a Single Parameter?
Evidence level: C
An assessor’s “instability” and “insensitivity” may simply be two facets of the same underlying capacity. If this is true, the dual-parameter framework designed to diagnose them separately would be entirely invalidated.
Main Body
When three assessors evaluate the same set of materials, two types of failure can occur: one is “drifting” (飘)—where the same batch of materials yields inconsistent results upon re-evaluation the next day, resulting in high noise; the other is “bluntness” (钝)—where the assessor fails to capture genuine differences between materials, exhibiting a low sensitivity slope and judging everything as “roughly the same.” In statistical models, these two issues can be expressed as two parameters: noise variance manages “drifting,” while sensitivity slope manages “bluntness.” Previous analyses treated them as two independent injected mechanisms, calculating their contributions separately.
However, this “independence” is a model assumption, not a measurement of real humans. Intuition in reality suggests the opposite: an assessor who drifts often does so precisely because they cannot grasp real differences between materials—”bluntness” and “drifting” may be two sides of the same underlying dimension. If they truly share a common origin, then all dual-parameter tables that separate these two asymmetries would be meaningless, as there is no independent slope term—they must collapse into a single-parameter “assessment capacity.”
Modern measurement science has actually prepared tools for this analysis. In Multi-Facet Rasch and Item Response Theory (IRT) models, the assessor’s “severity” and “discrimination” are inherently two parameters. Research using ordinal response models to estimate parameters for two raters on divergent thinking scoring data found that when inter-rater consistency is high, individual differences in severity and discrimination have little impact on final scores (PMID 40558819); meanwhile, the MRCP(UK) large-scale clinical exam analysis of 1259 examiners and 140,000 paired scores via Multi-Facet Rasch cleanly separated examiner severity differences from examinee ability differences—examiner severity accounted for only about 12% of the main effect variance (PMID 16919156). More critically, a Bayesian hierarchical latent trait model study simultaneously estimated rater bias (severity) and unreliability (noise) on large-scale essay scoring, finding that the impact on final scores stemmed primarily from unreliability rather than bias (PMID 29614129)—this result indicates noise operates at a larger magnitude, but it did not answer whether these two parameters are correlated across individuals.
This leaves a gap no one has filled: the coexistence of two parameters in a model does not mean they are independent at the individual level. Practical approaches to simultaneously estimate noise and sensitivity on the same cohort of subjects already exist (PMID 30870999); furthermore, the scale format itself can alter the structure of the ability dimension—ordered scales and visual analog scales yield different conclusions in perceptual evaluation (PMID 12417797)—so the dimensional structure must be empirically tested under specified conditions, not pieced together across studies.
There is ample reason to believe this correlation may be non-zero. First, both share the same underlying capacity: those who cannot grasp real differences resort to guessing, producing results that are both “blunt” (low slope) and “drifting” (re-evaluations mismatch). Second, both share the same cognitive resources: those with poor attention and working memory struggle to maintain stable standards and discern subtle differences. Third, training alters both simultaneously: systematically trained assessors tend to be both more stable and more accurate. Conversely, negative correlations may also arise—some individuals trade stability for sensitivity (overreacting to minor differences), while others trade sensitivity for stability (lumping all differences into the middle tier). These three scenarios correspond to three entirely different assessor management strategies, yet current literature contains no data distinguishing them.
Direct empirical testing is straightforward: ≥30 assessors × the same batch of ≥100 three-tier materials × each assessor re-evaluates 3 times, using multilevel models to simultaneously estimate each person’s noise variance and sensitivity slope, then calculating the correlation between these interpersonal estimates (using posterior distributions to provide credible intervals for the correlation coefficient). The core criteria are threefold: if the absolute correlation is less than 0.3, both parameters can be reported separately, validating the dual-parameter framework; if the correlation exceeds 0.6, real humans operate as single-parameter entities, and “bluntness” and “drifting” must merge into a single “assessment capacity” dimension, requiring all separate tables to be rewritten as single-parameter; if the correlation is negative (more stable individuals are actually more blunt), there exists a trade-off between efficiency and stability, necessitating two distinct assessment standards based on purpose rather than one unified ranking.
Implications for Yijing Studies: The traditional context treats “effort” (功夫) as a single dimension—deep mastery naturally yields both stability and accuracy. This paper transforms that single-dimensional intuition into a testable proposition: its validity depends on which of the three criteria the interpersonal correlation falls into.
Evidence and Limitations
Verified: Assessor severity and discrimination can be separated as two parameters from real large-scale data (PMID 16919156); bias and unreliability can be simultaneously estimated, with the latter having a greater impact on scores (PMID 29614129).
Partially Verified: Convergence and stability of dual-parameter estimation hold under IRT frameworks for high-consistency data (PMID 40558819); noise and sensitivity can be estimated in parallel on the same cohort of individuals (PMID 30870999).
Pending Verification: Interpersonal correlation between noise variance and sensitivity slope—all conclusions in this paper hinge on this number, which has never been directly measured; additionally, the modulation of dimensional structure by scale format (PMID 12417797) implies conclusions must be tied to specific scale types.
Further Reading
- Two Culprits Hidden in One Correlation Coefficient: Can Precision Ratio and Slope Ratio Be Separately Inverted?
- Common Offset and Non-Identifiability of Three-Tier Scales
- Research Pool: Resonant Consciousness Field
Evidence Level: C
References: PMID 29614129, PMID 40558819, PMID 16919156, PMID 30870999, PMID 12417797
Evidence Level: A=direct PubMed support, B=scientific analogy/indirect evidence, C=traditional consensus/preliminary exploration
L1 Network Status: This topic (assessor noise variance × interpersonal correlation with sensitivity slope) has not yet entered the L1 corresponding network—no edges exist in EDGE_TABLE_science-yixue.md or CORRESPONDENCE_NETWORK.md; the closest is the E-060 label reliability conflict (low diagnostic consistency among TCM physicians, PMID 30383398), which is adjacent but distinct. Therefore, this paper’s evidence level caps at C; all conclusions belong to pending verification designs and do not constitute verified mechanisms.
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply