Evidence level: B
同一批数据在多种校正方法和 q 阈值下,”显著漂移”的站点集合还剩下多少?用显著性集合的 Jaccard 重叠率裁决运气周期相位漂移结论是真信号还是统计假象。
正文
在 #702、#703 的两轮工作中,我们已经把”5-6 年运气尺度相位漂移”从单站分段小波推到了多站 meta 框架,用圆形统计(圆形均值、R̄、Rayleigh 检验)确认区域一致信号,并进一步用气候区、纬度、海拔三个调节变量解释 I²。但所有这些检验都默认了某个 p 阈值(多数研究采用 p < 0.05),而当一个数据集要同时跑多条周期带×多站点检验时——典型的物候学/运气学”筛选”问题——单一阈值就足以决定结论的命运。这就是为什么要做本轮的显著性稳健性裁决。
多重检验的经典陷阱是“显著性通胀”:每做一次检验,犯 I 类错误的概率就被叠加一次。Bonferroni 校正以 p = 0.05/m 直接抬高门槛,保守但代价是真实信号也被”一锅端”——Glickman 等(PMID 24831050)在 J Clin Epidemiol 上的方法学综述明确指出:在科学驱动的研究假设集合下,”FDR 控制几乎能保留与未校正方法相当数量的显著结果,而 Bonferroni 则把所有都判没了”。这一对比是 #704 一切检验设计的根本背景:我们若沿用 Bonferroni,相位漂移信号恐怕难逃”全军覆没”;若改用 BH-FDR(Benjamini-Hochberg 线性逐步上升法),则在控制期望假发现比例的前提下能保留更多真信号——这正是 Storey & Tibshirani 2003 年 PNAS 论文(PMID 12883005)提出的 q-value 框架的核心承诺。
具体到相位漂移场景,检验结构是双重的:周期带(5 年/6 年)×站点(N 站) 的笛卡儿积构成 M = (周期带数 × 站数) 个假设。我们对每一对做”两段时间相位差是否非零”的检验。原假设是相位差服从圆形均匀分布,备择是相位角集中于某个方向。这就引入了一个额外的问题:小波相位差天然有时序自相关,违背后的检验独立性假设。Phipson & Smyth 2010 年 Stat Appl Genet Mol Biol 上的论文(PMID 21044043)正是处理这种依赖结构下的精确 p 值计算——他们证明置换 p 值不应为零、必须用离散零分布的精确公式;这一方法为 #704 用自助置换构造依赖保留下的检验提供了方法学锚点。
裁决的核心指标:定义”显著漂移集” S(m, q) = 在校正方法 m、阈值 q 下被判为显著的 (站点, 周期带) 对的集合。我们跑 6 套组合(BH-FDR/Bonferroni/未校正 × q = 0.05/0.10),观察 S 的稳定性。具体步骤:(1) 任两套组合的 Jaccard 重叠率 J(A, B) = |A∩B| / |A∪B|;(2) 若所有 6 个集合的互相 Jaccard 平均 > 0.8,判定”显著漂移是稳健事实”;(3) 若 Jaccard 平均 < 0.5 且集合大小差异 > 3 倍,判定”阈值依赖”;(4) 中间情况则需看集合内的真实物理一致性。Jaccard 系数作为”集合相似度”的检验已被 Chung 等 2019 BMC Bioinformatics(PMID 31874610)严格建模,并提供了针对二元存在-缺席数据的 p 值与 FDR 估计流程,可直接搬用到显著集比较上。
回到本体论问题:阈值依赖最危险的不是”显著集合在变化”,而是”集合在不同方法下质性翻转”——比如某站在未校正下显著、用 Bonferroni 后不显著、S(BH-FDR) 包含它但 S(Bonferroni) 不包含。这种”擦边站”对综述结论最危险,因为它们最容易被选择性引用。我们的最终交付不应是”显著 vs 不显著”的二分,而应是”在所有 6 套检验下同时显著的稳健核心集”——这个集合才是真正可被复现、被引用、被作为下游证据的。Mathur & VanderWeele 2019 年在 J Am Stat Assoc(PMID 32981992)提出的 E-value 敏感性分析框架,本质就是为这种”跨越方法/阈值还能站得住”的结论背书。
工程层面还有三条”防滑”约束需要落到代码里:第一,输入的检验 p 值不应包含 ==0 的置换结果——Phipson & Smyth 已经证明随机抽取 m 次置换时的”零 p 值”系统性低估真实尾部概率约 1/m,这在 6 套组合交叉时会产生连锁假阳;稳妥的做法是用 Phipson-Smyth 校正的离散零分布公式重算所有 p 值后再输入 FDR/Bonferroni 层。第二,Bonferroni 的 m 必须定义为”同时跑的有效检验数”而非”理论上的笛卡儿积”——例如某些站点在 5 年与 6 年带上相位角重合度过高,这两个检验应合并而非独立报,否则 m 被人为放大、有效信号被过度惩罚。第三,BH-FDR 的 q 阈值 0.10 不是”宽松版”,而是对应 E-value 框架里”可接受假发现比例 10%”的明确承诺,写文章时要在表注里说清这一定义边界而不是模糊地写成”较宽松”。这三条加起来,才能让 6 套组合的 Jaccard 比较”比较的是科学”,而不是”比较的是各家方法学的小个性”。
证据与局限
已验证:FDR 控制作为 Bonferroni 替代的科学合理性,已被 Glickman 2014、Storey & Tibshirani 2003 等多家独立方法学综述共同背书(PMID 24831050/12883005);Jaccard 系数作为集合相似度度量有完整的 p 值/FDR 二元建模(PMID 31874610);置换检验在依赖结构下的精确 p 值有标准方法(PMID 21044043);E-value 框架为阈值-方法学稳健性提供了量化承载(PMID 32981992)。
部分验证:6 套组合交叉检验的具体接受阈值(Jaccard > 0.8 为稳健,< 0.5 为阈值依赖)目前是基于统计学常识的工程约定,而非已发表的”金标准”。需要后续实证研究(特别是合成数据 + 已知真信号的蒙特卡洛模拟)来标定这一阈值。
待验证:(1) 不同子集抽样(如去掉连续站点、换时间段)下稳健核心集是否仍能复现——若不能,则”看似稳健”实为样本特异;(2) 用替代检验(如基于 bootstrap 的相位差 CI 不跨零)得到的”显著集”是否与本框架重合——若两套正交方法都收敛到同一核心集,则证据最强;(3) 在中国 700+ 站点实测数据上跑这一交叉框架后,核心集的物理可解释性是否通过领域专家复审——比如东北-西藏-华东三组是否各占核心集的三分之一,或某种”差分模式”对应某种气候分型。
相关阅读
证据等级: B
参考文献: PMID 24831050, PMID 12883005, PMID 21044043, PMID 31874610, PMID 32981992
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
Phase Shift Significance: Robust Fact or Threshold Dependency: Cross-Validation of FDR Correction and q-Value
Evidence level: B
How many sites remain classified as showing “significant drift” across multiple correction methods and q-thresholds? Use the Jaccard overlap rate of significant sets to adjudicate whether conclusions regarding luck cycle phase drift represent true signals or statistical artifacts.
Main Text
In the two rounds of work #702 and #703, we escalated “5-6 year luck-scale phase drift” from a single-station segmented wavelet approach to a multi-station meta-framework. We confirmed regional consistency signals using circular statistics (circular means, R̄, Rayleigh tests) and further explained I² using three moderator variables: climate zones, latitude, and altitude. However, all these tests default to a specific p-threshold (most studies use p < 0.05). When a dataset undergoes multiple periodic bands × multi-station tests simultaneously—a typical “screening” problem in phenology/Yijing Studies—a single threshold can determine the fate of the conclusion. This necessitates the current Significance Robustness Adjudication.
The classic trap of multiple testing is “significance inflation”: every additional test accumulates the probability of Type I error. Bonferroni correction raises the threshold directly to p = 0.05/m; while conservative, it risks “cooking the book” on true signals. A methodological review by Glickman et al. (PMID 24831050) in J Clin Epidemiol explicitly states that under a set of research hypotheses driven by science, “FDR control retains almost as many significant results as uncorrected methods, whereas Bonferroni eliminates all of them.” This contrast provides the fundamental background for all testing designs in #704: if we adhere to Bonferroni, phase drift signals may be “wiped out”; if we switch to BH-FDR (Benjamini-Hochberg linear step-up method), we can retain more true signals while controlling the expected false discovery proportion. This aligns with the core promise of the q-value framework proposed by Storey & Tibshirani in their 2003 PNAS paper (PMID 12883005).
Specifically in the phase drift scenario, the testing structure is dual: the Cartesian product of periodic bands (5-year/6-year) × stations (N sites) constitutes M = (number of bands × number of sites) hypotheses. We perform a test for each pair to determine if the phase difference between two time periods is non-zero. The null hypothesis assumes the phase difference follows a circular uniform distribution, while the alternative assumes phase angles concentrate in a specific direction. This introduces an additional issue: wavelet phase differences naturally exhibit temporal autocorrelation, violating the independence assumption underlying standard tests. The paper by Phipson & Smyth (PMID 21044043) in Stat Appl Genet Mol Biol addresses precise p-value calculation under such dependency structures. They demonstrate that permutation p-values should not be zero but must use exact formulas for discrete zero distributions; this method provides the methodological anchor for #704’s use of bootstrap permutation to construct tests preserving dependency.
Core Adjudication Metric: Define the “significant drift set” S(m, q) as the collection of (station, band) pairs deemed significant under correction method m and threshold q. We run six combinations (BH-FDR/Bonferroni/uncorrected × q = 0.05/0.10) to observe the stability of S. Specific steps: (1) Calculate the Jaccard overlap rate J(A, B) = |A∩B| / |A∪B| for any two sets; (2) If the average Jaccard across all six sets exceeds 0.8, adjudicate “significant drift as a robust fact”; (3) If the average Jaccard is < 0.5 and set size differences exceed 3-fold, adjudicate “threshold dependency”; (4) Intermediate cases require examining true physical consistency within the sets. The Jaccard coefficient, used as a measure of set similarity, has been rigorously modeled by Chung et al. (PMID 31874610) in BMC Bioinformatics, providing p-values and FDR estimation procedures for binary presence-absence data, which can be directly applied to significant set comparisons.
Returning to the ontological question: the most dangerous aspect of threshold dependency is not merely that “significant sets change,” but that “sets undergo qualitative flips under different methods”—e.g., a station is significant under uncorrected testing, becomes non-significant under Bonferroni, yet is included in S(BH-FDR) but excluded from S(Bonferroni). Such “borderline stations” pose the greatest risk to review conclusions as they are most susceptible to selective citation. Our final deliverable should not be a binary “significant vs. non-significant” classification, but rather the “robust core set significant across all six testing frameworks”—this is the only set truly reproducible, citable, and suitable as downstream evidence. The E-value sensitivity analysis framework proposed by Mathur & VanderWeele (PMID 32981992) in J Am Stat Assoc essentially provides endorsement for conclusions that “stand firm across methods and thresholds.”
Three “slip-prevention” constraints must be implemented in the code: First, input permutation p-values should not contain ==0 results—Phipson & Smyth have proven that randomly drawing m permutations systematically underestimates the true tail probability by approximately 1/m, which can cause cascued false positives when crossing six combinations. The safe approach is to recalculate all p-values using the Phipson-Smyth corrected discrete zero distribution formula before inputting them into FDR/Bonferroni layers. Second, the m in Bonferroni must be defined as the number of “simultaneously run effective tests” rather than “theoretical Cartesian products”—for instance, if phase angles overlap excessively at certain stations between the 5-year and 6-year bands, these two tests should be merged rather than reported independently; otherwise, m is artificially inflated, and true signals are overly penalized. Third, the BH-FDR q-threshold of 0.10 is not a “looser version” but corresponds to the explicit commitment in the E-value framework of an “acceptable false discovery proportion of 10%”; article notes must clarify this definitional boundary rather than vaguely writing “relatively loose.” Together, these three constraints ensure the Jaccard comparison across six combinations compares “science” rather than “methodological idiosyncrasies.”
Evidence and Limitations
Verified: The scientific rationale for FDR control as a Bonferroni alternative is endorsed by multiple independent methodological reviews, including Glickman 2014 and Storey & Tibshirani 2003 (PMID 24831050/12883005); the Jaccard coefficient as a set similarity metric has complete p-value/FDR binary modeling (PMID 31874610); permutation tests under dependency structures have standard methods for precise p-values (PMID 21044043); and the E-value framework provides quantitative carrying capacity for threshold-method robustness (PMID 32981992).
Partially Verified: The specific acceptance thresholds for the six-combination cross-validation (Jaccard > 0.8 for robust, < 0.5 for threshold dependency) are currently engineering conventions based on statistical common sense rather than published “gold standards.” Subsequent empirical research (particularly Monte Carlo simulations with synthetic data and known true signals) is needed to calibrate these thresholds.
To Be Verified: (1) Whether the robust core set remains reproducible under different subset sampling (e.g., removing consecutive stations, changing time periods)—if not, “apparent robustness” may be sample-specific; (2) Whether “significant sets” derived from alternative tests (e.g., phase difference CI not crossing zero based on bootstrap) overlap with this framework—if two orthogonal methods converge to the same core set, evidence is strongest; (3) Whether, after running this cross-framework on实测 data from over 700 sites in China, the physical interpretability of the core set passes domain expert review—for example, whether the Northeast-Tibet-East China groups each occupy one-third of the core set or if a certain “differential pattern” corresponds to a specific climate classification.
Related Reading
- Adjudication of “Inter-station Consistency” in Multi-station Luck Cycle Phase Drift
- Adjudication of “Climate Zone Attribution” for Inter-station Heterogeneity in Phase Drift
- Research Pool · Spatiotemporal Frequency Topology
Evidence Level: B
References: PMID 24831050, PMID 12883005, PMID 21044043, PMID 31874610, PMID 32981992
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply