Evidence level: C
上一轮那条精确律 P_k=(δ/s_B)^(k−2),全部两百万次抽样都建立在「剩余质量均匀摊给每个次档」的假设上——而真人恰恰不这么打分。假设一换,安全份额会朝两个方向都错,错多少至今没人算过。
正文
先把前情说清。多档有序量表的 kappa 校正问题上,前文已经证明:三档量表之所以有一个干净的闭式安全区,是因为它把自由度压成了一维;从四档起,安全区要乘上一个小于一的收缩因子,而且每加一档乘一次,乘到第五档基本归零——这就是 P_k=(δ/s_B)^(k−2) 这条精确律。但它有一个从未被审视的地基:所有模拟都假设,两位判读者在主档一致之外的那部分分歧质量,是按均匀 Dirichlet 摊给各次档的。
真人打分恰恰不是这样。心理学测量文献早就反复提示,量表档数的实际收益取决于反应分布的真实形状:跨多个心理测量指标的比较发现,二至四档量表在信度、效度与区分力上显著较差,七档之后趋于平坦(PMID 10769936);一项 1358 人被随机分派到 2 至 11 档的实验显示,二至五档精度明显衰减而六档以上无额外增益——这条「档数收益曲线」之所以长成这样,正是因为次档质量的真实分布决定了有效维数(PMID 30869956);对 Rosenberg 自尊量表的同类操纵也得到一致结论(PMID 36050908)。分布形状不是细节,它就是有效维数本身。
于是问题可以拆成三类真实构型。第一类是集中型:材料有陡峭的难度梯度时,次档质量高度集中在紧邻主档的那一档——「轻度改善」占九成、「明显改善」几乎为零。此时次档差的实际维数远低于 k−1,几何上退化回近似一维的三档情形,安全份额会显著高于收缩因子的预测。第二类是两端型:质量分给两个极端次档,中间空掉,维数名义上用满但间隔被拉大。第三类是交叉型:两位判读者在不同次档上有系统性偏好——一位爱用「轻度」、一位爱用「明显」——这恰恰是维数被真正用满的情形,安全份额会低于预测。
这三类构型对应三种完全不同的处置。若集中型构型下安全份额比因子预测高出三倍以上,说明现行样本量表在最常见的「材料难度梯度陡」场景下系统性过保守,样本量可以整体下调——这对用四档、五档量表做一致性研究的人是直接的省钱消息。若交叉型构型下安全份额低于预测一半,则样本量表必须补一条前置条件:两位判读者的次档偏好要先做一致性预检,预检不合格的,样本量直接翻倍——因为偏好交叉恰恰意味着维数被用满、几何上最危险。而若三类构型的安全份额极差超过一个量级,那 P_k=(δ/s_B)^(k−2) 就不能作为单一公式使用,必须给出按构型分档的三张表,并在使用前先判定自己落在哪一类。
可做的数值实验并不复杂:把次档分配从均匀 Dirichlet 换成上述三类构型,各扫一个浓度参数,重算 k=3 到 6 的安全份额,与均匀假设的因子预测逐点比对,输出「偏离超过 20% 所需的集中度阈值」。这个阈值就是那张样本量表的适用边界——低于它,旧表照用;高于它,换表。
判读者不是均匀随机数生成器,这从来不是哲学姿态,而是几何事实:均匀假设给的是维数被平均使用时的答案,而真实评分总是把质量堆在某些方向上。堆得越偏,有效的维数越少,安全区越宽;堆得越「交叉」,维数用得越满,安全区越窄。一张不知道自己假设了什么形状的样本量表,在两个方向上都可能错,而它现在连自己错在哪一边都说不出来。
证据与局限
已验证:三个 PMID 全部为档数效应的直接实验或荟萃证据(10769936 / 30869956 / 36050908),「分布形状决定有效维数」在心理测量学内有坚实支撑。
部分验证:三类构型(集中/两端/交叉)在真实评分数据中的出现频率未量化,本文只给出机制论证与实验设计。
待验证:① 集中型构型下安全份额相对因子预测的实际放大倍数;② 交叉型构型下的压低幅度;③ 「偏离 20%」的集中度阈值位置。三者均需按文中数值方案跑完后才能定表。
相关阅读
L1 网络状态:该主题尚未进入 L1 对应网络(CORRESPONDENCE_NETWORK.md 与 EDGE_TABLE_science-yixue.md 中无「κ 样本量 × 次档分布构型敏感度」对应边;最近为 E-060 标签前提「标签信度」边,状态 conflict:中医医师间诊断一致性在开放人群中「低」(PMID 30383398),且「中低一致性 + 大样本 + 类别数多」时一致性统计精度显著下降——与本主题的方法学关切相邻,但非同一条边)。
证据等级: C
参考文献: PMID 10769936, PMID 30869956, PMID 36050908
证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
That contraction factor is calculated under the assumption of \”sub-grade uniform spreading\”: real readers are not uniformly distributed, and sensitivity to the distribution shap…
Evidence level: C
The precise law $P_k=(\delta/s_B)^{k−2}$ from the previous round, which underpins all two million simulations, is built upon the assumption that “remaining mass is uniformly spread to each sub-category.” Real human raters do not score this way. Changing this assumption causes the safe margin to err in both directions; the magnitude of this error has not yet been calculated.
Main Text
Let us clarify the context first. Regarding the kappa correction for multi-tier ordered scales, it has been proven previously that the three-tier scale possesses a clean closed-form safe zone because it reduces degrees of freedom to one dimension. From four tiers onward, the safe zone must be multiplied by a contraction factor less than one, and each additional tier requires another multiplication, reducing the safe zone to nearly zero by the fifth tier—this is the precise law $P_k=(\delta/s_B)^{k−2}$. However, this law rests on a foundation that has never been examined: all simulations assume that the portion of disagreement mass between two raters, beyond their agreement on main categories, is distributed uniformly via Dirichlet across sub-categories.
Real human scoring does not follow this pattern. Psychological measurement literature has repeatedly indicated that the actual benefit of scale tiers depends on the true shape of the response distribution: cross-comparisons across multiple psychometric indicators show that two-to-four-tier scales are significantly inferior in reliability, validity, and discrimination power, while seven or more tiers tend to flatten (PMID 10769936); an experiment randomly assigning 1,358 participants to scales with 2 to 11 tiers found that precision clearly degraded from two to five tiers while six or more tiers yielded no additional gain—this “tier-benefit curve” takes this shape precisely because the true distribution of sub-category quality determines the effective dimension (PMID 30869956); similar manipulation of the Rosenberg Self-Esteem scale also yielded consistent conclusions (PMID 36050908). Distribution shape is not a detail; it is the effective dimension itself.
Thus, the problem can be decomposed into three real configurations. The first is the concentrated type: when materials have a steep difficulty gradient, sub-category quality is highly concentrated in the tier immediately adjacent to the main category—「slight improvement」accounts for ninety percent, while「marked improvement」is nearly zero. In this case, the actual dimension of sub-category differences is far lower than $k−1$, geometrically degenerating back to the approximately one-dimensional three-tier scenario, causing the safe margin to be significantly higher than predicted by the contraction factor. The second is the two-end type: quality is assigned to two extreme sub-categories while the middle is left empty; the dimension is nominally fully utilized but intervals are stretched. The third is the crossed type: two raters have systematic preferences for different sub-categories—one favors「slight」while the other favors「marked」—which is precisely when the dimension is truly fully utilized, causing the safe margin to fall below predictions.
These three configurations correspond to three completely different treatments. If, under the concentrated configuration, the safe margin exceeds the factor prediction by more than three times, it indicates that current sample scales are systematically overly conservative under the most common scenario of “steep material difficulty gradient,” allowing the sample size to be reduced overall—a direct cost-saving message for those using four-tier or five-tier scales for consistency studies. If, under the crossed configuration, the safe margin falls below half the prediction, then the sample scale must add a prerequisite condition: consistency pre-checks for sub-category preferences between two raters must be performed first; if they fail the pre-check, the sample size must double—because crossed preferences mean the dimension is fully utilized and geometrically most dangerous. And if the safe margin variance across the three configurations exceeds one order of magnitude, then $P_k=(\delta/s_B)^{k−2}$ cannot be used as a single formula; instead, three tables stratified by configuration must be provided, and users must determine which configuration applies before use.
The numerical experiments that can be conducted are not complex: replace the uniform Dirichlet sub-category allocation with the three configurations above, sweep each concentration parameter, recalculate safe margins for $k=3$ to $6$, and compare point-by-point with the factor prediction under the uniform assumption, outputting “concentration thresholds required for deviation exceeding 20%.” This threshold defines the applicability boundary of the sample scale—below it, use the old table; above it, switch tables.
Raters are not uniform random number generators; this is never a philosophical stance but a geometric fact: the uniform assumption gives the answer when dimensions are used on average, while real scoring always piles quality onto certain directions. The more skewed the pile, the fewer effective dimensions and the wider the safe zone; the more “crossed” the pile, the fuller the dimension usage and the narrower the safe zone. A sample scale that does not know what shape it assumes can err in both directions, and currently cannot even say which direction it errs.
Evidence and Limitations
Verified: All three PMIDs provide direct experimental or meta-analytic evidence of tier effects (10769936 / 30869956 / 36050908); “distribution shape determines effective dimension” has solid support in psychometrics.
Partially Verified: The frequency of occurrence of the three configurations (concentrated/two-end/crossed) in real scoring data has not been quantified; this paper provides only mechanistic argumentation and experimental design.
To Be Verified: ① the actual amplification factor of safe margin relative to the factor prediction under concentrated configuration; ② the suppression magnitude under crossed configuration; ③ the location of the “20% deviation” concentration threshold. All three must be determined after running the numerical scheme described in this paper.
Further Reading
- The beautiful 1+2E for three tiers is a coincidence caused by “only one direction remaining”
- Is “floating judgment” the same as “dull judgment”: will two-parameter diagnosis collapse into single-parameter?
- Research pool: Resonant consciousness field
L1 Network Status: This topic has not yet entered the corresponding L1 network (no edge for “κ sample size × sub-category distribution configuration sensitivity” in CORRESPONDENCE_NETWORK.md and EDGE_TABLE_science-yixue.md; the most recent edge is E-060 labeled “label reliability,” with status conflict: diagnostic consistency among TCM practitioners is “low” in open populations (PMID 30383398), and statistical precision of consistency drops significantly under “medium-low consistency + large sample + many categories”—adjacent to this methodological concern but not the same edge).
Evidence Level: C
References: PMID 10769936, PMID 30869956, PMID 36050908
Evidence Level: A = direct PubMed support; B = scientific analogy/indirect evidence; C = traditional consensus/preliminary exploration
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply