改连续打分救不了三档:省下的量化误差本就不计入读者分歧,真正吃亏的是另一种场景

Written by

in

Language

Evidence level: C

三档的量化误差是”共享”的,它压低了 ICC 却从不进入读者间分歧;因此换成连续打分,读数会掉,真实精度也不一定赚——账要分两个口径算,不能合成一笔。

正文

三档设计的读数天花板不够高,怎么办?最顺手的答案是”改连续打分”——档位是离散的,把连续量切成三格必然丢信息,而连续打分不丢。听上去是无损升级。

但这个推理漏了一件事:量化误差和读者间分歧并不是同一笔账。 误差至少有两种来源。一种是”尺子本身刻得粗”,它对所有判读者一视同仁——大家都用同一组切点,所以这部分误差不会让两个判读者互相矛盾,只会让所有人一起偏离真值。另一种是”每人心里那把尺子不一样”——零点偏高或偏低、量程压得紧或放得松,这部分才会变成判读者之间的分歧。

读者间一致性指标(ICC、kappa 一类)只看得见后一种。它把”两人读数差多少”当成误差的全部,而共享的那部分粗刻度,因为两人都有、且方向一致,相减时互相抵消,于是一分钱也不算进分歧里。这就是问题所在:三档的量化误差一直在替三档”藏拙”,让它的 ICC 显得比真实精度好看。 换成连续打分,这把共享的粗尺子没了,ICC 会立刻掉下来——不是因为连续打分更差,而是它不再享有那层隐身。

这笔账算出来是多少

把这个问题形式化:设真实状态服从标准正态,三档用等频切点分成三格,每格用格内条件期望作代表值。这样算出的共享量化误差方差是 0.206 个标准差平方(约 0.45 个标准差),明显大于均匀量化理想值”步长平方除以十二”给出的 0.062——因为三格的边缘格开口很大,条件期望对极端值的代表性差。

现在让两位判读者各带一份方差 0.25(即 0.5 个标准差)的知觉噪声。这时三档的真实误差是 0.467,读数算出 0.793。若改用连续打分,并刻意让它达到完全相同的真实误差 0.467,它的读者间分歧就等于 0.467 全部(没有共享部分可抵消),读数只有 0.681。同一份真实精度,读数差了 0.111。 这 0.111 不是连续打分的缺陷,是三档那 0.206 从未被计数。

反过来看更刺人:若要求连续打分也报出 0.793,它必须把真实误差压到 0.261——也就是比三档准 44%,才能打平同一个数字。所以”读数相同”绝不等于”精度相同”,两者相差一个共享量化误差的距离。

那连续打分到底亏不亏

这才是原题要的判决。模型里,连续打分给每位判读者的读数加上一个固定的零点偏移(标准差 sb)和一个量程缩放偏差(sg),再加知觉噪声。三者独立,真实误差就是三者平方和。设知觉噪声 0.5、量程偏差 0.1,连续打分要优于同噪声水平的三档,零点偏移必须小于 0.453 个标准差。

这个门槛不算苛刻,但也不宽松。文献报告过的判读者严厉度差异,通常在一个到两个档位之间摆动;三档的档步长是 0.86 个标准差,换句话说,只要判读者的习惯零点差异超过半个档位,连续打分就白换了。 知觉噪声升到 0.7 时门槛掉到 0.293,几乎只剩三分之一档的余量。

文献站哪一边

实测证据是分裂的,而且分裂得很规整。骨折愈合的影像评分研究里,十位临床医生对 58 例(含 14 张重复片)同时用分类量表和 10 厘米视觉模拟线评分,连续打分的判读者间相关是 0.49,分类量表的加权 kappa 只有 0.25(PMID 29351044)。听觉感知的嗓音偏离度研究也同向:六位专家评 205 例,视觉模拟线的判读者间一致性高于数字分类(PMID 26222946)。但鱼类活力评分的研究结论相反:三位评分者对 304 尾鱼同时用三档分类和连续模拟线打分,连续打分没有改善活力指数与延迟死亡的拟合,作者直接建议”用三档、别用连续”(PMID 28704390)。更早的精神科症状量表研究则给出更冷的答案:26 个条目分别用分类和模拟量表评分,再把模拟结果重切成 2、3、4 档,一致性水平不受量表形式影响(PMID 515274)。

三个方向都有,说明”连续 vs 分类”本身没有单一答案——它取决于该场景里零点偏移相对量化误差谁更大。这正是本轮算出的门槛在起作用,而不是哪一派更对。

还有一条更硬的路

真正值得记的不是”哪个更好”,而是操作后果:只用一致性指标来选量表形式,会系统性地偏向粗量表。 因为粗量表有一块误差不计入指标,分数天然虚高。临床能力评估的研究已反复撞到这件事——评分者严厉度差异会明显改变学生成绩与及格判定,而给评分者反馈自己的评分分布、试图校准,效果并不显著,严厉度还会随时间漂移(PMID 28587597);用多面 Rasch 模型拆解 173 名学生的临床技能评分,能直接量出严厉度差异、评分不一致、以及”只用高档位”这种量程收缩,四类错误各自独立存在(PMID 17693619)。

所以选量表形式的正确问法不是”哪个读数高”,而是”我的判读者零点差异有多大,我的量化误差有多大,哪笔账更贵“。前一笔记进指标,后一笔不进——必须分开算,相加才是真实精度。

证据与局限

已验证: 在明确设定的生成模型下(三档等频切点、条件期望解码、判读者零点与量程为常数偏差),”共享量化误差不计入读者间分歧、连续打分无此豁免”是可解析验证的恒等式,不是经验猜测;仿真与解析式在 0.001 量级内吻合。文献层面,”量表形式改变一致性读数”这一现象已有多个独立场景的实测报告(PMID 29351044、26222946、28704390、515274)。

部分验证: 零点偏移的实际量级。文献确认判读者严厉度差异普遍存在且难以校准(PMID 28587597、17693619),但没有给出可直接换算到”标准差单位”的分布参数,因此文中 0.453 这个门槛只能作为敏感性分析的界,不能当既定常数使用。

待验证: ①本文的量化误差计算依赖”等频切点 + 条件期望解码”,实际量表若用不等频切点或简单取档序号,q2 会变,门槛随之移动;②文献里连续打分占优的三项研究(影像、嗓音)与不占优的两项(鱼类、精神科)之间,是否真能用零点偏移量级解释,还是另有第三种机制(如任务本身的可分性),目前无法从摘要层面裁决;③共享量化误差是否也影响重测信度、以及影响幅度是否与读者间口径一致,尚未计算。

相关阅读

证据等级: C
参考文献: PMID 29351044, PMID 28704390, PMID 515274, PMID 28587597, PMID 17693619, PMID 26222946

L1 网络状态:该主题尚未进入 L1 对应网络。最近邻边为 E-060 标签前提(标签信度),其状态原文为「冲突(未裁决,且新增两处单位错配/精度区间错配)」——KB-292 报中医师间诊断一致性在开放人群中「低(low)」(PMID 30383398),而同系列指出加权系数在「中低一致性 + 大样本 + 类别数多」时精度显著下降(PMID 33853549)。该边只登记「标签信度够不够用」,未登记「量表形式如何改变信度读数的记账口径」;L1 网络中「零点」「量程」「连续打分」三词命中数均为 0。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Continuous Scoring Cannot Save Three Tiers: The Saved Quantization Error Is Already Excluded from Reader Disagreement, and the Real Loss Occurs in a Different Scenario

Evidence level: C

The quantization error of three tiers is “shared”; it depresses ICC but never enters reader disagreement. Therefore, switching to continuous scoring causes the reading score to drop, and true precision is not necessarily gained—the accounts must be calculated on two different scales, not merged into one.

Body

The ceiling of the reading score for three-tier designs is insufficient; what do we do? The most convenient answer is “switch to continuous scoring”—since tiers are discrete, chopping a continuous quantity into three bins inevitably loses information, whereas continuous scoring does not. This sounds like an upgrade without loss.

But this reasoning misses one thing: quantization error and reader disagreement are not the same account. Error has at least two sources. One is “the ruler itself is coarsely marked,” which applies equally to all judges—everyone uses the same set of cutpoints, so this part of the error does not cause two judges to contradict each other; it merely causes everyone to deviate from the true value together. The other is “each person’s internal ruler is different”—zero points shifted high or low, ranges compressed tight or stretched loose; this part becomes the disagreement between judges.

Reader consistency indices (ICC, kappa, etc.) only see the latter. They treat “how much do two readings differ” as the entirety of error, while the shared part of coarse scaling, since both judges have it and it points in the same direction, cancels out when subtracted, so not a cent is counted into disagreement. This is the crux of the problem: the quantization error of three tiers has been hiding the shortcomings of three tiers, making its ICC appear better than its true precision. Switching to continuous scoring removes this shared coarse ruler, so ICC immediately drops—not because continuous scoring is worse, but because it no longer enjoys that layer of invisibility.

How much does this account come to?

Formalizing the problem: suppose the true state follows a standard normal distribution, three tiers use equal-frequency cutpoints to divide into three bins, and each bin uses the conditional expectation within the bin as its representative value. The resulting shared quantization error variance is 0.206 standard deviation squared (approximately 0.45 standard deviations), clearly larger than the uniform quantization ideal value of “step squared divided by twelve” yielding 0.062—because the edge bins of three tiers have large openings, and conditional expectations poorly represent extreme values.

Now suppose two judges each carry perceptual noise with variance 0.25 (i.e., 0.5 standard deviations). Then the true error of three tiers is 0.467, and the reading score calculates to 0.793. If we instead use continuous scoring and deliberately make it achieve the exact same true error of 0.467, its reader disagreement equals the full 0.467 (no shared part to cancel), so the reading score is only 0.681. The same true precision yields a reading difference of 0.111. This 0.111 is not a defect of continuous scoring; it is because the 0.206 from three tiers was never counted.

Looking at it the other way is even more striking: if we demand continuous scoring also report 0.793, it must suppress the true error to 0.261—44% more accurate than three tiers—to match the same number. Thus “same reading” never means “same precision”; they differ by the distance of shared quantization error.

So does continuous scoring actually lose out?

This is the judgment the original question seeks. In the model, continuous scoring adds to each judge’s reading a fixed zero-point offset (standard deviation sb) and a range scaling bias (sg), plus perceptual noise. These three are independent; true error is the sum of their squares. Suppose perceptual noise is 0.5 and range bias is 0.1; for continuous scoring to outperform three tiers at the same noise level, the zero-point offset must be less than 0.453 standard deviations.

This threshold is not overly strict, but it is not loose either. Literature reports of judge severity differences typically swing between one and two tiers; the step size of three tiers is 0.86 standard deviations, meaning if the judges’ habitual zero-point difference exceeds half a tier, switching to continuous scoring is pointless. When perceptual noise rises to 0.7, the threshold drops to 0.293, barely leaving room for one-third of a tier.

Where does the literature stand?

Empirical evidence is split, and split in a regular way. In fracture healing imaging scoring studies, ten clinicians simultaneously used categorical scales and 10 cm visual analog scales on 58 cases (including 14 repeated images); the correlation between continuous scorers was 0.49, while the weighted kappa of categorical scales was only 0.25 (PMID 29351044). Auditory perception studies of voice deviation also align: six experts rated 205 cases, and consistency among visual analog scorers exceeded that of digital categorizers (PMID 26222946). But fish vitality scoring studies concluded the opposite: three raters scored 304 fish simultaneously using three-tier categorization and continuous analog lines, and continuous scoring did not improve the fit of vitality index to delayed mortality; authors directly advised “use three tiers, don’t use continuous” (PMID 28704390). An earlier psychiatry symptom scale study gave an even colder answer: 26 items were scored using both categorical and analog scales, then the analog results were re-chopped into 2, 3, or 4 tiers, and consistency levels were unaffected by scale form (PMID 515274).

All three directions exist, showing that “continuous versus categorical” itself has no single answer—it depends on whether the zero-point offset relative to quantization error is larger in that scenario. This is precisely where the threshold calculated above comes into play, not which side is more correct.

There is still a harder path

What truly matters to remember is not “which is better,” but the operational consequence: using consistency indices alone to choose scale form systematically favors coarse scales. Because coarse scales have a portion of error not counted in the index, their scores are naturally inflated. Clinical skills assessment studies have repeatedly bumped into this—rater severity differences significantly alter student scores and pass/fail judgments, while giving raters feedback on their own score distributions to attempt calibration has not significantly improved results; severity still drifts over time (PMID 28587597); decomposing clinical skills scores of 173 students using the many-facet Rasch model directly measures severity differences, rater inconsistency, and “using only high tiers” range contraction; these four errors exist independently (PMID 17693619).

Thus the correct question for choosing scale form is not “which reading is higher,” but “how large are my raters’ zero-point differences, how large is my quantization error, which account costs more.” The first enters the index; the second does not—must be calculated separately, added together for true precision.

Evidence and Limitations

Verified: Under clearly specified generative models (three tiers with equal-frequency cutpoints, conditional expectation decoding, raters’ zero points and ranges as constant biases), “shared quantization error does not enter reader disagreement, continuous scoring lacks this exemption” is an analytically verifiable identity, not empirical guesswork; simulations and analytical formulas agree within 0.001 magnitude. At the literature level, “scale form changes consistency readings” has been reported in multiple independent scenarios (PMID 29351044, 26222946, 28704390, 515274).

Partially Verified: The actual magnitude of zero-point offsets. Literature confirms rater severity differences are ubiquitous and hard to calibrate (PMID 28587597, 17693619), but does not provide distribution parameters directly convertible to “standard deviation units”; therefore the threshold of 0.453 in this text can only serve as a sensitivity analysis bound, not a fixed constant.

To Be Verified: ① This paper’s quantization error calculation relies on “equal-frequency cutpoints + conditional expectation decoding”; if actual scales use unequal-frequency cutpoints or simple bin indices, q2 changes and the threshold shifts accordingly; ② among the three studies where continuous scoring prevailed (imaging, voice) and two where it did not (fish, psychiatry), whether zero-point offset magnitude alone can explain the difference, or whether a third mechanism (such as task divisibility itself) is at play, cannot be adjudicated from abstracts alone; ③ whether shared quantization error also affects test-retest reliability, and if so, whether the effect magnitude matches the reader-disagreement scale, has not been computed.

Further Reading

Evidence Level: C
References: PMID 29351044, PMID 28704390, PMID 515274, PMID 28587597, PMID 17693619, PMID 26222946

L1 Network Status: This topic has not yet entered the L1 corresponding network. The nearest edge is the E-060 label premise (label credibility), whose original status reads “Conflict (unresolved, and two additional unit mismatches/precision interval mismatches added)“—KB-292 reports that diagnostic consistency among TCM practitioners in open populations is “low” (PMID 30383398), while the same series indicates weighted coefficients suffer significant precision decline under “medium-low consistency + large sample + many categories” (PMID 33853549). This edge only registers “whether label credibility suffices,” not “how scale form changes the accounting scale of credibility readings”; in the L1 network, hit counts for “zero point,” “range,” and “continuous scoring” are all zero.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *