用偏度峰度去分辨「切点有没有被训练动过」,这套判据一开动就是反的:离散分档上位置移动本身就改形状

Written by

in

Language

Evidence level: C

题目假设「只移动位置不动切点,分级分布的形状不变,所以偏度和峰度可以用来验形状」——这条假设在连续分布上成立,但判读用的是五档离散格子,位置一移动,格子边界就被重新分配,形状必然变。判据因此把两种机制判反了。

一、题目想解决什么,以及它的主张

前序工作(#1421)证明了一个约束:在多面 Rasch 模型里,评分者的位置参数 b 与切点组的整体平移完全共线,六点 profile 扫描下 ΔNLL 全为零,要估计就必须外生假定「两轮共享同一套切点」。#1458 想把这条外生假定变成可检验的:如果训练只动了 b,那么同一个评分者在训练前后两轮的档级频次分布,应当是「同一形状沿潜在轴滑了一段」,偏度和峰度不该变;如果动的是切点间距(把某一档切得更细),形状本身会改变,偏度能捕捉到。判据说得很明确:一阶矩差显著、二至四阶矩差不显著,则「只动 b」成立。

这个想法在教科书式的连续正态模型里是对的。问题在于,评分者交出来的不是连续值,是 1 到 5 这五个格子。

二、硬算一遍:位置一移动,形状就变

取一套不均匀的五级切点(间距比约 0.9 : 0.9 : 1.0,贴近真实判读习惯),固定间距、只让位置从 −1.0 走到 +1.0,逐点算五档概率和前四阶矩:

位置 均值 方差 偏度 超额峰度
−1.0 4.079 0.813 −0.757 +0.016
−0.5 3.633 1.018 −0.408 −0.408
0.0 3.138 1.128 −0.126 −0.592
+0.5 2.629 1.118 +0.159 −0.658
+1.0 2.144 0.977 +0.512 −0.486

切点间隔一个都没有动,偏度却从 −0.757 走到 +0.512,跨度 1.27。 峰度从 +0.016 走到 −0.658,跨度 0.67。原因很朴素:偏度和峰度是连续分布的形状描述量,它们假定支撑集不随位置移动。一旦格子固定、分布滑过去,大量样本从 5 档掉到 4 档、从 2 档升到 3 档,格子把连续分布切成了不同形状的多项式——形状当然变。

所以题目的判据前提不成立。「二至四阶矩不变」不是一个可以拿来当零假设的性质,它本身就是被违反的。

三、判据不只失效,而且是反向的

更麻烦的是失效的方向。取一个真实「只动 b」的场景(位置 −0.9 走到 +0.6,间距全程固定 1.0),算偏度差:

真·只动b:Δ偏度 = +0.901,抽样标准差(n=400)= 0.122,z = 7.37

判据规定「偏度差显著 ⇒ 切点被动过」。于是在这个切点一动没动的场景里,判据以 z=7.37 的强度宣布「切点被动了」。样本越大越确信,n=1600 时 z=14.7——加大样本只会让误判更坚定。

再取一个真实「动切点」的场景(位置不动,间距从 1.0 拉到 1.3):

真·动切点:Δ偏度 = +0.012,抽样标准差(n=400)= 0.123,z = 0.10

判据规定「偏度差不显著 ⇒ 只动 b」。于是切点确实被动了,判据却以 z=0.10 宣布「没动」。而且这不是样本量不够——把 n 从 100 加到 1600,z 只从 0.05 涨到 0.21,永远够不到显著线。因为间距变化对偏度的真效应本身就只有 0.012,比 n=400 时的抽样噪声小一个数量级。

两种机制在矩上的指纹对照:

真实机制 Δ均值 Δ偏度 Δ峰度
只动 b −1.467 +0.901 −0.547
只动切点 +0.013 +0.012 +0.342
两者都动 −1.340 +0.718 −0.266

判据选的「偏度」恰好是区分力最差的量:它对位置极度敏感(0.901),对切点几乎瞎(0.012)。判据把最灵敏的通道接到了错误的因上,把最钝的通道当成了主判据。

四、那到底哪里不可识别,哪里可以识别

这里要把两件事分开,混在一起会得出错误结论。

不可识别是真的。 自由切点模型有 5 个参数(位置 b 加 4 个切点),单轮五档只给 4 个独立比例,自由度 −1。多起点拟合同一组观测,能给出 b = 0.0、0.5、1.0、−0.5 等无限多组解,残差全在 1e−16 量级。这是严格不可识别,#1421 的判断没错。

但不可识别的来源不是「位置与间距分不开」,而是「切点形状完全未知」。 上面那四组解里,切点间距的绝对值随 b 变,但相对形状(间距比 0.964 : 0.964 : 1.071)每一组都完全一致。也就是说:形状是可从单轮识别的,只有绝对尺度不可识别。

一旦把切点形状固定下来——比如假定等距,只留位置 b 和一个间距 s——参数降到 2 个,单轮 4 个比例给出自由度 2,模型变成可识别。多起点拟合五组不同的真值 (−0.8,0.7)、(0,1.0)、(0.9,1.3)、(−1.2,1.8)、(0.4,0.5),全部唯一收敛到真值,残差 1e−21 到 1e−31。

所以正确的表述是:训练有没有改变切点形状,这件事本身是可检验的,但前提是先接受一个切点形状的约束模型(如等距),再用模型比较去检验这个约束。 不能靠无约束的矩去绕开它。

五、换成似然比检验,判据才站得住

正确的做法是嵌套模型比较:零模型让两轮共享同一个间距 s(3 参数),备择模型让两轮各自有 s(4 参数),用似然比统计量做卡方检验(自由度 1)。同一套数据,功效是这样的:

每读者病例数 检出「切点形状漂移」的把握
60 0%
120 1%
240 8%
480 36%
960 91%

而真只动 b 时的误报率,n=120 和 n=480 都是 0%(名义水平 5%)。也就是说这个检验是「保守但正确」——不会冤枉只动 b 的读者,只是在样本小时抓不到真的漂移。

关键结论:n=120 例/读者的常规判读研究,对切点形状漂移的检验力只有 1%,等同于没有检验。 20 名读者各 120 例合并起来,把握仍然是 0%——合并读者不能救回单读者层面的形状漂移,因为漂移是读者特异的方向,合并会互相抵消。

要做这件事,单读者需要 900 例以上。这个数字直接决定了「切点有没有被训练动过」这个问题在现有数据集上根本没有答案,而不是「答案是不显著」。

六、这条结论的边界

第一,这里的计算基于正态潜变量加固定切点的分档模型。真实判读数据的潜变量未必正态,厚尾或双峰会改变矩的具体数值,但不会改变方向性:离散化之后位置移动必然改形状,这条与分布族无关(只要格子数有限、格子宽度有限)。

第二,等距切点是一个强假定,本文用它只是为了让模型可识别,不是说真实切点等距。更稳的做法是把切点形状参数化(比如用两个参数控制不对称性),再做嵌套比较。

第三,本文只处理了「形状漂移」这一种机制。训练还可能同时改变评分者使用的档级数(category collapse),那会让某些格子概率趋零,此时似然比的渐近卡方近似会失效,需要用精确检验或参数自助法。

证据与局限

已验证:
– 离散分级下固定切点只移动位置,偏度必然改变:本文网格计算,位置从 −1.0 到 +1.0 偏度跨 1.27,峰度跨 0.67
– 多面 Rasch 模型中评分者位置参数与时间特异严重度漂移是已建模对象,且时间独立性假设被明确批评为低效(PMID 36284065)
– 评分者宽严(leniency/stringency)是可估计、可跨时间复现的真实效应(PMID 16919156:1259 名考官 142,030 条评分,12% 主效应方差来自考官宽严;PMID 8322872 早期证实评分者严重度显著)
– 小样本下 kappa 类系数的置信区间不可靠、需要足够测试例数(PMID 20473138)

部分验证:
– 多面 Rasch 的五类评分者效应(含 central tendency 与 leniency)在模型内可分离(PMID 14523257),但这是连续潜变量层面,不是离散档级分布层面
– 训练确实能改变判读行为(PMID 21899969 专用培训改变观察者间变异;PMID 37127220 国际间乳腺密度判读差异),但这两项都报告系数的变化,未报告档级分布形状的变化

待验证:
– 本文的偏度真效应量(0.012)依赖正态潜变量假定,尚未在真实五档判读数据上用半参数方法复核
– 「单读者 900 例才能检出形状漂移」这个数字是在间距从 1.0 拉到 1.3(效应量偏大)条件下算的,真实效应更小时所需样本还要更大
– 档级塌缩(category collapse)下的检验方法未做功效评估

相关阅读

证据等级: C
参考文献: PMID 36284065, PMID 16919156, PMID 8322872, PMID 14523257, PMID 20473138, PMID 21899969, PMID 37127220

L1 网络状态:该主题尚未进入 L1 对应网络。CORRESPONDENCE_NETWORK.md(1881 行)全文检索「切点」仅 1 处(E-060 条目内 AC2 定切点的实操说明,与本主题无关)、「分档」1 处(KB-496 整数分箱风险,属 GDV 参数分箱,不同层)、「偏度」0 处、「峰度」0 处、「平移」0 处、「位置参数」0 处、「可识别」0 处、「severity」0 处;EDGE_TABLE_science-yixue.md(63 行、59 条概念边)无任何一致性系数或分档分布层概念。故本主题无对应边,证据等级封顶 C。

证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Using skewness and kurtosis to determine whether the cut point has been trained, this criterion is inverted: moving the position on the discrete binning inherently alters the sh…

Evidence level: C

The premise assumes that “moving only the location while keeping cutpoints fixed preserves the graded distribution’s shape, so skewness and kurtosis can verify the shape”—a hypothesis valid for continuous distributions but flawed when applied to five discrete bins. Shifting the location reassigns bin boundaries, inevitably altering the shape. Consequently, the criterion reverses the two mechanisms it aims to distinguish.

I. What the Problem Aims to Solve and Its Claim

Prior work (#1421) established a constraint: in the Multidimensional Rasch model, the rater location parameter $b$ is perfectly collinear with a uniform shift of the entire cutpoint group. Under a six-point profile scan, $\Delta\text{NLL}$ remains zero; estimation requires the exogenous assumption that “two rounds share the same cutpoints.” #1458 seeks to make this exogenous assumption testable: if training shifts only $b$, the frequency distribution of grades for the same rater across two rounds should represent “the same shape sliding along the latent axis,” leaving skewness and kurtosis unchanged; if training alters cutpoint spacing (making one grade finer), the shape itself changes, and skewness captures this. The criterion states clearly: a significant first-moment difference with insignificant second through fourth moment differences implies “only $b$ moved.”

This logic holds in textbook continuous normal models. The problem lies in the fact that raters do not provide continuous values but rather fall into five discrete bins (1 to 5).

II. Hard Calculation: Moving Location Alone Alters Shape

Using an uneven five-level cutpoint scheme (spacing ratios approximately 0.9 : 0.9 : 1.0, reflecting real-world grading habits), we fix the spacing and shift the location from −1.0 to +1.0, calculating five-grade probabilities and the first four moments at each step:

Location Mean Variance Skewness Excess Kurtosis
−1.0 4.079 0.813 −0.757 +0.016
−0.5 3.633 1.018 −0.408 −0.408
0.0 3.138 1.128 −0.126 −0.592
+0.5 2.629 1.118 +0.159 −0.658
+1.0 2.144 0.977 +0.512 −0.486

With cutpoint intervals unchanged, skewness shifts from −0.757 to +0.512, a span of 1.27. Kurtosis moves from +0.016 to −0.658, a span of 0.67. The reason is straightforward: skewness and kurtosis describe the shape of continuous distributions, assuming the support set does not shift with location. Once bins are fixed and the distribution slides over them, large numbers of samples drop from bin 5 to bin 4 or rise from bin 2 to bin 3. The bins slice the continuous distribution into different-shaped polynomials—the shape necessarily changes.

Thus, the premise of the criterion fails. “Second through fourth moments unchanged” is not a property to serve as the null hypothesis; it is itself violated.

III. The Criterion Not Only Fails but Operates in Reverse

More troubling is the direction of failure. Consider a true “only $b$ moves” scenario (location shifts from −0.9 to +0.6, spacing fixed at 1.0 throughout), calculating skewness difference:

True·only b moves: Δskewness = +0.901, sampling SD (n=400) = 0.122, z = 7.37

The criterion stipulates “significant skewness difference ⇒ cutpoints moved.” In this scenario where cutpoints did not move at all, the criterion declares with $z=7.37$ that “cutpoints were altered.” Larger samples increase certainty: at $n=1600$, $z=14.7$—increasing sample size only strengthens the misjudgment.

Now consider a true “cutpoints moved” scenario (location fixed, spacing stretched from 1.0 to 1.3):

True·cutpoints moved: Δskewness = +0.012, sampling SD (n=400) = 0.123, z = 0.10

The criterion stipulates “insignificant skewness difference ⇒ only $b$ moved.” Thus, where cutpoints indeed moved, the criterion declares with $z=0.10$ that “they did not move.” This is not due to insufficient sample size—raising $n$ from 100 to 1600 increases $z$ only from 0.05 to 0.21, never reaching significance. The spacing change exerts a true effect on skewness of only 0.012, an order of magnitude smaller than sampling noise at $n=400$.

Fingerprints of the two mechanisms on moments:

True mechanism ΔMean ΔSkewness ΔKurtosis
Only $b$ moves −1.467 +0.901 −0.547
Only cutpoints move +0.013 +0.012 +0.342
Both move −1.340 +0.718 −0.266

Skewness—the metric chosen by the criterion—is precisely the least discriminative quantity: highly sensitive to location (0.901) but nearly blind to cutpoints (0.012). The criterion connects the most sensitive channel to the wrong cause and treats the least sensitive channel as the primary criterion.

IV. What Is Unidentifiable and What Is Identifiable

We must separate these two issues; conflating them yields erroneous conclusions.

Unidentifiability is genuine. The free cutpoint model has 5 parameters (location $b$ plus 4 cutpoints). A single round with five grades provides only 4 independent proportions, yielding degrees of freedom −1. Fitting multiple starting points to the same observations yields infinitely many solutions ($b = 0.0, 0.5, 1.0, −0.5$, etc.), with residuals on the order of $10^{-16}$. This is strict unidentifiability; #1421’s judgment is correct.

However, the source of unidentifiability is not “location and spacing inseparable,” but rather “cutpoint shape completely unknown.” Among the four solution sets above, the absolute values of cutpoint spacing vary with $b$, yet the relative shape (spacing ratios 0.964 : 0.964 : 1.071) remains identical across all solutions. Thus: shape is identifiable from a single round; only absolute scale is unidentifiable.

Once cutpoint shape is fixed—for example, assuming equal spacing and retaining only location $b$ and one spacing $s$—parameters drop to 2. A single round with four proportions yields degrees of freedom 2, rendering the model identifiable. Fitting five distinct true values $(−0.8,0.7), (0,1.0), (0.9,1.3), (−1.2,1.8), (0.4,0.5)$ converges uniquely to the true values, with residuals ranging from $10^{-21}$ to $10^{-31}$.

Therefore, the correct statement is: whether training altered cutpoint shape is itself testable, but only after accepting a constrained cutpoint shape model (e.g., equal spacing) and then using model comparison to test that constraint. One cannot bypass this by relying on unconstrained moments.

V. Likelihood Ratio Testing Makes the Criterion Viable

The proper approach is nested model comparison: the null model assumes both rounds share the same spacing $s$ (3 parameters), while the alternative model allows each round its own $s$ (4 parameters). A chi-square test using the likelihood ratio statistic (degrees of freedom 1) yields the following power for the same dataset:

Number of cases per rater Power to detect “cutpoint shape drift”
60 0%
120 1%
240 8%
480 36%
960 91%

Meanwhile, the false positive rate when only $b$ moves is 0% for both $n=120$ and $n=480$ (nominal level 5%). Thus, this test is “conservative yet correct”—it never falsely accuses a rater of only $b$ moving, though it fails to detect true drift with small samples.

Key conclusion: the conventional criterion of 120 cases per rater has only 1% power to detect cutpoint shape drift, equivalent to no test at all. Combining 20 raters with 120 cases each still yields 0% power—merging raters cannot recover shape drift at the individual level because drift is rater-specific in direction, and merging causes cancellation.

To achieve this, a single rater requires more than 900 cases. This number directly determines that the question “whether cutpoints were altered by training” has no answer on existing datasets, rather than “the answer is insignificant.”

VI. Boundaries of This Conclusion

First, these calculations assume normal latent variables with fixed cutpoints. True grading data may not be normally distributed; heavy tails or bimodality alter specific moment values but do not change directionality: after discretization, location shifts inevitably alter shape, regardless of the distribution family (provided finite bins and finite bin widths).

Second, equal-spacing cutpoints is a strong assumption; we use it solely to render the model identifiable, not because true cutpoints are equally spaced. A more robust approach parameterizes cutpoint shape (e.g., using two parameters controlling asymmetry) and then performs nested comparison.

Third, this paper addresses only “shape drift.” Training may also alter the number of grades used by raters (category collapse), causing certain bin probabilities to approach zero. In such cases, the asymptotic chi-square approximation of the likelihood ratio fails, requiring exact tests or parametric bootstrapping.

Evidence and Limitations

Verified:
– Under fixed cutpoints in discrete grading, moving only location necessarily changes skewness: our grid calculation shows skewness spanning 1.27 and kurtosis spanning 0.67 as location shifts from −1.0 to +1.0
– In the Multidimensional Rasch model, rater location parameters and time-specific severity drift are modeled objects, with the assumption of time independence explicitly criticized as inefficient (PMID 36284065)
– Rater leniency/stringency is an estimable, reproducible effect across time (PMID 16919156: 1,259 examiners graded 142,030 items, with 12% of main effect variance attributable to examiner leniency; PMID 8322872 early confirmed significant rater severity)
– Under small samples, confidence intervals for kappa-type coefficients are unreliable and require sufficient test cases (PMID 20473138)

Partially Verified:
– The five-grade rater effects in Multidimensional Rasch (including central tendency and leniency) are separable within the model (PMID 14523257), but this pertains to continuous latent variables, not discrete grade distributions
– Training indeed alters grading behavior (PMID 21899969: specialized training reduces inter-observer variability; PMID 37127220: international differences in breast density grading), but both report changes in coefficients without reporting changes in grade distribution shape

Pending Verification:
– Our true skewness effect size (0.012) depends on the normal latent variable assumption and has not been re-verified using semi-parametric methods on real five-grade grading data
– The figure “900 cases per rater needed to detect shape drift” was calculated under spacing stretched from 1.0 to 1.3 (large effect size); true effects smaller would require even larger samples
– Testing methods under category collapse have not undergone power analysis

Related Reading

Evidence Level: C
References: PMID 36284065, PMID 16919156, PMID 8322872, PMID 14523257, PMID 20473138, PMID 21899969, PMID 37127220

L1 Network Status: This topic has not entered the L1-corresponding network. A full-text search of CORRESPONDENCE_NETWORK.md (1,881 lines) for “cutpoints” yields only one occurrence (AC2 operational instructions for setting cutpoints within entry E-060, unrelated to this topic), “grading” one occurrence (KB-496 integer binning risk, belonging to GDV parameter binning at a different layer), “skewness” zero occurrences, “kurtosis” zero occurrences, “shift” zero occurrences, “location parameter” zero occurrences, “identifiable” zero occurrences, “severity” zero occurrences; EDGE_TABLE_science-yixue.md (63 lines, 59 concept edges) contains no consistency coefficients or grade distribution layer concepts. Hence, this topic has no corresponding edge, and evidence level is capped at C.

Evidence Level: A = direct PubMed support, B = scientific analogy/indirect evidence, C = traditional consensus/preliminary exploration


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *