15% 这条线是量出来的还是拍的:联合适用域边界对阈值选择的敏感性

Written by

in

Language

Evidence level: C

任何画在网格上的「适用域」边界,都是一条人造的等值线:阈值挪一挪,域就跟着挪——阈值本身有多稳,域的结论才有多硬。

正文

上一篇文章(mse-balance-rho-m-joint-applicability-single-line-boundary)给出了一个联合适用域:在 ρ×m×df 的仿真网格上,把逐格偏差不超过 15% 的格子圈起来,作为「单行判决公式可以放心用」的区域。这个交付物看起来客观——每格偏差都是算出来的。但圈域的那条线不是网格自己长出来的,是设计者选的:为什么是 15%,不是 10% 或 20%?这一问就问到了要害——如果换一个同样说得过去的阈值,域的形状大不一样,那「域内」这个结论就不是数据的属性,而是阈值的属性。

这在测量学里不是新问题。诊断检验研究早就发现:用数据挑出来的「最优 cutoff」会让灵敏度和特异度系统性地过于乐观,样本越小偏得越狠,想要稳健的阈值估计需要相当大的样本量(PMID 18258670)。近年更进一步的思路是:阈值的选择根本不该只看统计指标,还要看选错阈值 downstream 的代价——把不同阈值下的后果摆出来比较,结论才立得住(PMID 36970849)。换句话说,阈值线不是从数据里「读」出来的,是「定」出来的;定了之后,它对结果的拉动必须单独交账。

适用域研究里同样如此。化学信息学的「适用域」定义明确指出:画边界本质上是在做离群判定,而边界画在哪里,直接决定多少对象被圈进来、多少被排除(PMID 27492083)。更细的基准测试发现,不同的域定义方法之间是覆盖率与可靠性的权衡——域画得宽松,覆盖广但错判多;画得严,错判少但几乎不剩什么可用区(PMID 32756326)。这些研究共同的教训是:域边界是一个带旋钮的交付物,报告域而不报告旋钮位置及其敏感性,等于交付了半张图。

画在网格上的边界还有一个特殊结构值得单独盯:角格。网格四个角对应最极端的参数组合——ρ 最低、m 最大那一角,往往是偏差曲线最陡的地方,也是三档阈值之间距离拉得最开的地方。如果说域内部各格在三档阈值下归属一致还算平常,角格的归属翻来翻去才是真正的警报:它说明边界位移不是均匀的「整体膨胀收缩」,而是从最脆弱的角先破。所以三档敏感性检验除了算总位移率,还要单列角格归属表——四个角在三档下各归域内还是域外,一翻就知道这条边界从哪里开始不可信。

落到我们这条线上,检验方案是现成的:#1562 已经产出 ρ×m×df 网格的逐格偏差数据,基准锚点取 ρ=0.4、m≤8。在同一批数据上分别按 10%、15%、20% 三档阈值画三条域边界,然后量两件事:其一,格位移率——从 15% 边界挪到 10% 或 20% 边界时,有多少格的归属翻转;其二,角格归属——网格四个角(最极端的参数组合)在三档阈值下是否一致。判据很直白:若 10% 与 20% 两条边界之间的位移达到半张网格的量级,说明「15% 域内」对阈值高度敏感,分档表的启用区必须连阈值带宽一起报——即交付物从「这条线内可用」改写成「这条线±X 格内可用、具体取决于你接受多大偏差」;若三档边界几乎重合、位移不到一格,说明域的形状对阈值不敏感,15% 这条线才算「量出来的」而非「拍的」,分档表可以放心用。

这个检验还有一层更实际的意义:域图是给下游用的。下游拿到「15% 域内」会当成一条硬边界来切数据,他们不知道这条线本身有 ±5 个百分点的带宽。上游不报敏感性,下游就会把软边界当硬边界用——这是测量链条上最常见的静默失效方式。与其等下游误用,不如在上游就把阈值的带宽标定出来,和域图一起交付。

证据与局限

已验证:数据驱动选 cutoff 会系统性高估判别精度,且样本越越小偏得越狠(PMID 18258670);适用域边界的定义方式直接决定覆盖与错判的权衡,不同定义结论不同(PMID 27492083、PMID 32756326);把阈值的下游代价纳入选择已是成熟做法(PMID 36970849)。

部分验证:「阈值敏感性应作为域图交付物的一部分」是从上述文献直接类推到本线的方法学要求,逻辑成立但未在本线数据上跑过。

待验证:10%/15%/20% 三档边界在本线 ρ×m×df 网格上的实际位移率与角格归属——这是纯仿真可算的,等 #1562 网格复用即可出数;出数之前,15% 域只能当参考图,不能当硬边界用。

相关阅读

证据等级: C
参考文献: PMID 18258670, PMID 36970849, PMID 27492083, PMID 32756326

证据等级:A=有PubMed直接支撑,B=科学类比/间接证据,C=传统文化共识/初步探索

L1 网络状态:该主题尚未进入 L1 对应网络(CORRESPONDENCE_NETWORK.md 与 EDGE_TABLE_science-yixue.md 中无「阈值选择敏感性 × 适用域边界位移」对应边)。最近邻边为 E-061「呼吸节律(0.1 Hz 谱峰)↔ Mayer 波 / 自主神经谱」,层级 L2、状态 open——原文摘录:「0.1 Hz 四来源(Mayer 脊髓源 / 呼吸谐波倍频 / Traube-Hering 呼吸相位 / 拍频)已找到可操作分离判据(屏气 vs 定速呼吸)……L2(操作→观测量→阈值三件套已具备,判别实验待做)」。该边只覆盖呼吸节律与自主神经谱的机制归属,不覆盖测量学阈值敏感性,本文按 C 级处理。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

Is the 15% threshold line measured or captured: sensitivity of joint applicability domain boundaries to threshold selection

Evidence level: C

Any “applicability domain” boundary drawn on a grid is an artificial contour line: shifting the threshold shifts the domain accordingly—the stability of the conclusion depends on how stable the threshold itself is.

Body

The preceding article (mse-balance-rho-m-joint-applicability-single-line-boundary) established a joint applicability domain: on the ρ×m×df simulation grid, cells with per-cell deviations not exceeding 15% are circled as the region where “the single-row judgment formula can be safely applied.” This deliverable appears objective—each cell’s deviation is calculated. However, the line defining this circled region was not generated by the grid itself; it was selected by the designer: why 15%, and not 10% or 20%? This question strikes at the crux—if a different, equally plausible threshold is chosen, the shape of the domain changes dramatically; thus, the conclusion that “the region is within the domain” reflects properties of the threshold rather than the data.

This is not a new problem in metrology. Diagnostic testing research has long established that an “optimal cutoff” derived from data systematically overestimates sensitivity and specificity, with smaller samples yielding greater bias; robust threshold estimation requires substantial sample sizes (PMID 18258670). More recent approaches emphasize that threshold selection should not rely solely on statistical metrics but must account for the downstream costs of choosing an incorrect threshold—comparing consequences across different thresholds yields more robust conclusions (PMID 36970849). In other words, the threshold line is not “read” from the data but “set”; once set, its influence on results must be accounted for separately.

Applicability domain studies similarly face this issue. The definition of “applicability domain” in cheminformatics explicitly states that drawing boundaries is essentially outlier detection, and where the boundary is drawn directly determines how many objects are included versus excluded (PMID 27492083). Finer benchmarking reveals that different domain definition methods involve a trade-off between coverage and reliability: loose boundaries yield broad coverage but more misclassifications, while strict boundaries reduce errors but leave little usable area (PMID 32756326). The common lesson is: the domain boundary is a tunable deliverable; reporting the domain without specifying the knob position and its sensitivity is akin to delivering only half a map.

The boundary drawn on the grid has another special structure worth separate attention: corner cells. The four corners of the grid correspond to the most extreme parameter combinations—specifically, the corner where ρ is lowest and m is highest often represents the steepest part of the deviation curve and where the distance between the three thresholds is greatest. If consistent assignment of internal grid cells across all three thresholds is considered normal, the fluctuating assignment of corner cells serves as a true warning: it indicates that boundary displacement is not uniform “overall expansion/contraction” but rather breaks first at the most vulnerable corners. Therefore, the three-threshold sensitivity test must not only calculate the total displacement rate but also separately tabulate corner cell assignments—checking whether each of the four corners falls within or outside the domain under each threshold. A flip in corner assignment immediately reveals where the boundary becomes unreliable.

Applied to our specific case, the testing protocol is ready: #1562 has already produced per-cell deviation data for the ρ×m×df grid, with baseline anchors at ρ=0.4 and m≤8. Using the same dataset, three domain boundaries are drawn at thresholds of 10%, 15%, and 20%. Two metrics are then measured: first, the grid displacement rate—how many cells change assignment when moving from the 15% boundary to either the 10% or 20% boundary; second, corner cell assignment—whether the four corners (the most extreme parameter combinations) remain consistent across all three thresholds. The criteria are straightforward: if the displacement between the 10% and 20% boundaries spans half the grid, it indicates that “within the 15% domain” is highly sensitive to threshold choice; the tiered table must then report the usable region together with the threshold bandwidth—rephrasing the deliverable from “usable within this line” to “usable within ±X cells of this line, depending on the maximum deviation you accept.” Conversely, if the three boundaries nearly overlap with displacement less than one cell, the domain shape is insensitive to the threshold; the 15% line is truly “measured” rather than “arbitrarily set,” and the tiered table can be used with confidence.

This test also carries practical significance: the domain map is intended for downstream use. Downstream users interpreting “within the 15% domain” will treat it as a hard boundary for data slicing, unaware that this line itself has a ±5 percentage point bandwidth. If upstream does not report sensitivity, downstream will mistakenly treat the soft boundary as hard—this is the most common silent failure mode in measurement chains. Rather than waiting for downstream misapplication, the upstream should calibrate and report the threshold bandwidth alongside the domain map.

Evidence and Limitations

Verified: Data-driven selection of the cutoff systematically overestimates discriminative accuracy, and smaller samples yield greater bias (PMID 18258670); the definition of domain boundaries directly determines the trade-off between coverage and misclassification, leading to divergent conclusions under different definitions (PMID 27492083, PMID 32756326); incorporating downstream costs of the threshold into selection is an established practice (PMID 36970849).

Partially Verified: The requirement that “threshold sensitivity should be delivered as part of the domain map” is a methodological extension derived directly from the above literature to this line of inquiry; the logic holds, but it has not yet been tested against data from this specific line.

Pending Verification: The actual displacement rate and angular grid membership for the three boundary tiers (10%/15%/20%) on the ρ×m×df grid of this line—these are purely simulated values and can be computed once the #1562 grid is reused; until numerical results are available, the 15% domain must be treated as a reference diagram and not used as a rigid boundary.

Related Reading

Evidence Level: C
References: PMID 18258670, PMID 36970849, PMID 27492083, PMID 32756326

Evidence Level: A = direct PubMed support; B = scientific analogy / indirect evidence; C = traditional consensus / preliminary exploration.

L1 Network Status: This topic has not yet entered the corresponding L1 network (no edge linking “threshold-selection sensitivity × applicability-domain boundary displacement” exists in CORRESPONDENCE_NETWORK.md and EDGE_TABLE_science-yixue.md). The nearest neighbor edge is E-061 “respiratory rhythm (0.1 Hz spectral peak) ↔ Mayer wave / autonomic spectrum”, at level L2, status open — excerpted from the original text: “Four sources of 0.1 Hz (Mayer spinal source / respiratory harmonic submultiples / Traube-Hering respiratory phase / beat frequency) have yielded an operational separation criterion (breath-hold vs constant-rate breathing) … L2 (the triad of operation → observable → threshold is in place; discriminative experiments remain to be conducted).” This edge covers only the mechanistic attribution of respiratory rhythm and autonomic spectrum, not metrological threshold sensitivity; therefore this article is treated at Level C.


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *