Evidence level: C
同一批人的三模态方向投票,判「通过 / 失败」用的 10/12、ICC 0.3、相关 0.5、功效 30% 四个门槛,到底哪来的?追到底会发现:其中三个是「惯例」不是「规定」,而且零假设本身设得太保守——换成等效性检验后,「三角验证」的结论形式可能根本不可达。
正文
气功外气研究里,「三角验证」是个常用话术:同一构造,用 %AB、φ、X5、CoV 四个参数、12 个格子同时测,格子同向率超过某条线就算「验证通过」。但这条线是怎么定出来的,几乎没人问过。本文要做的,就是把这几个数字的出处一个个挖出来,看它们是「规定」(有方法学依据、有出处)还是「惯例」(大家这么用,没人为它负责)。
先看第一个数字:ICC ≥ 0.3 算「有信度」。这个数在测量学界其实有一整条可查的来源链——最权威的出处是 COSMIN(健康测量工具遴选共识标准)。COSMIN 对信度的评级标准是 ICC ≥ 0.70 为「充分」、< 0.70 为「不充分」、没报告为「不确定」(PMID 17161752,Terwee 2007 年发表在 J Clin Epidemiol 的《Quality criteria were proposed for measurement properties of health status questionnaires》)。注意:0.3 根本不在 COSMIN 的档位里。学界流传的「0.3 及格」来自更早的心理测量文献里「临床 vs 研究用途」的粗略分层(研究用途可放宽),但那不是 COSMIN 的判定标准。所以 ICC 0.3 这条线,是「惯例」——它对应的是「研究用途」的宽口径,不是「这个测量能不能用」的判定。
第二个数字:相关 ≥ 0.5。这个数同样能追到 COSMIN 的构造效度标准:与「测同一构造」的工具相关 ≥ 0.50 视为充分(同一份 COSMIN 文件里有明确条款,PMID 17161752)。也就是说「相关 0.5」是构造效度的收敛标准,被拿来当「两模态方向一致」的线——语义上是近亲,但不是一回事。收敛效度问「两个工具测的是不是同一个东西」,投票一致性问「两个格子方向朝不朝同一个方向」,后者没有 COSMIN 条款。
第三个数字:功效 30%。这个最离谱。功效(power)在统计学的任何标准里都是越高越好,公认下限是 80%(这是统计教材级的规定,不是惯例——Cohen 的经典框架里 0.80 是「想要但常达不到」的目标)。30% 作为「判据门槛」出现在这里,是把「探测功效」误用成了「通过线」:30% 其实意味着「有真效应时也只有三成概率能检出」,这个数字当门槛,等于允许七成概率漏报。它既不是规定,也不是惯例,更像是口误。
第四个数字:10/12(83% 同向率)。这是唯一一个真正由二项分布算出来的数:12 格、每格独立、零假设同向率 0.5,双侧 α=0.05 时,≥10 格同向才显著(P≈0.019)。它的问题是零假设本身:二项检验假设每一格都在「独立抛硬币」,可 12 个格子测的是同一个构造,同向率低于 0.9 都不该算「验证通过」——零假设设成 0.5,等于把「同向率 0.6-0.8 的弱证据」也放进「可能通过」的区间。这正是#1266 那篇「12 格折成 6.3 格甚至 4.8 格」指控的统计侧根源:格子不独立,二项框架的整体假设就已经打折。
那替代方案是什么?最干净的一条是等效性检验(TOST,two one-sided tests)。TOST 的原始出处是 Schuirmann 1987 年在 J Pharmacokinet Biopharm 上的论文(PMID 3450848):把零假设从「无差异」翻过来,改成「差异太大不可接受」,上下两条单侧检验都拒绝,才能宣布「等效」。用在投票判据上就是:不问「同向率是否显著大于 0.5」,而问「同向率是否显著落在 [0.7, 0.9] 的等效区间内」——等效界取多宽,直接决定 12 格需要几格同向。如果等效界取 0.1(即同向率 0.8±0.1 算通过),所需格数会显著高于 10/12,甚至超过现有可行上限——那「三角验证通过」这个结论形式本身就不可达,只能退而逐模态独立报告。
还有一层:投票计数的口径混乱。系统综述方法学里有实证记录:非元分析的合成方法(包括投票计数)被广泛使用却极少被说明(PMID 36758885,2023 年 J Clin Epidemiol 调查),投票计数在少量研究下异质性检验表现很差(PMID 10412966,Am J Epidemiol 1999)。这三条合起来的意思是:把「方向投票」当验证结论的载体,方法学本身就不稳——这跟符号检验在平票不确定时功效大跌(PMID 20974291)是同一个病根:投票法的功效对「平票格怎么算」极其敏感。
落到研究设计上,可做一次纯方法学标定(无人体数据即可):把零假设 p0 从 0.5 扫到 0.9、格数从 6 扫到 65、门槛从「精确二项」换成 TOST,输出三张对照表——(a) 不同 p0 下的最小同向格数 (b) 达到 80% 功效所需格数 (c)「判失败 / 未检出 / 判通过」三区的位置;再用 ≥40 人实测同向率作锚点校准 p0。三条核心判据:若 p0 提到 0.7 后 9/12 就能显著,则现有「7/12 判失败」的结论在更合理零假设下翻成「无法判断」,此前所有基于 p0=0.5 的失败判定须整体重估;若 TOST 等效界 0.1 时所需格数 > 现有可行上限,「三角验证通过」结论形式不可达,须改逐模态独立报告;若三个惯例数能追到方法学出处,则 #1188 的自认应改成引用具体 GRADE/COSMIN 条款。
证据与局限
已验证: 二项检验的保守性有方法学文献支撑——投票计数在少量研究下的缺陷(PMID 10412966)与平票不确定性对符号检验功效的打击(PMID 20974291)都是公开文献结论;COSMIN 的质量标准(ICC ≥ 0.70、相关 ≥ 0.50)有明确出处(PMID 17161752)。
部分验证: TOST 等效性检验作为「支持无效应」的标准做法有原始论文(PMID 3450848)和后续方法学综述(PMC5502906)支撑,但它在方向投票这类「格子非独立」场景的适用性,只有理论推演、没有实证标定——本文的核心判据(p0 提到 0.7 后门槛落到几格)需要蒙特卡洛模拟才能落地。
待验证: ① 12 格同向率的真实分布——用 ≥40 人有经验者实测,校准 p0 该取 0.5 还是 0.7/0.9;② TOST 等效界取 0.1 时所需格数与可行上限的对照表——纯数值模拟即可产出;③ 三个惯例数(ICC 0.3 / 相关 0.5 / 功效 30%)在库内既往文章中的实际使用范围——需要一次库内全量统计,判断「惯例」是否已经渗进结论措辞。
L1 网络状态:该主题尚未进入 L1 对应网络。CORRESPONDENCE_NETWORK 逐词检索「ICC / 门槛 / 功效 / TOST / 等效 / 二项 / 同向 / 投票 / 符号检验 / 异质性 / 判据」均无对应边;EDGE_TABLE_science-yixue 反抽表亦无此边——「方向投票的判据门槛出处」目前是网络外的方法学规则层议题,本文为该方法学缺口补上第一层标定。
相关阅读
证据等级: C
参考文献: PMID 3450848, PMID 17161752, PMID 20974291, PMID 36758885, PMID 10412966
免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。
83% How much of that line is regulation and how much is convention: tracing the origin of criterion thresholds and exploring alternatives
Evidence level: C
In studies of the same group using three modalities, four thresholds are applied to judge “pass/fail”: 10/12, ICC ≥ 0.3, correlation ≥ 0.5, and power ≥ 30%. Where do these come from? Tracing them reveals that three are conventions rather than regulations, and the null hypothesis itself is set too conservatively—switching to equivalence testing may make the “triangulation” conclusion form unattainable.
Body
In Qigong external energy research, “triangulation” is a common phrase: the same construct is measured simultaneously using four parameters (%AB, φ, X5, CoV) across 12 cells, and “verification passed” is declared if the rate of same-direction cells exceeds some threshold. Yet how this threshold was determined has rarely been questioned. This article traces the origins of these numbers to determine whether they are “regulations” (methodologically grounded with cited sources) or “conventions” (widely used without specific responsibility).
First number: ICC ≥ 0.3 counts as “reliable.” In the measurement community, there is a complete traceable chain of sources for this. The most authoritative source is COSMIN (Consensus-based Standards for the selection of Measurement Instruments). COSMIN’s reliability rating standards define ICC ≥ 0.70 as “sufficient,” < 0.70 as “insufficient,” and no reporting as “uncertain” (PMID 17161752, Terwee 2007 published in J Clin Epidemiol titled “Quality criteria were proposed for measurement properties of health status questionnaires”). Note: 0.3 is not in COSMIN’s tiers at all. The widely circulated “0.3 passes” comes from an earlier psychometric literature’s rough stratification of “clinical vs research use” (research use allows relaxation), but this is not COSMIN’s decision criterion. Therefore, ICC ≥ 0.3 is a convention—it corresponds to the broad “research use” scope, not a judgment of whether “this measurement can be used.”
Second number: correlation ≥ 0.5. This also traces back to COSMIN’s construct validity standards: tools correlated with those measuring the same construct ≥ 0.50 are considered sufficient (explicitly stated in the same COSMIN document, PMID 17161752). Thus “correlation 0.5” is the convergence criterion for construct validity, repurposed as the threshold for “two modalities agreeing in direction.” Semantically they are close relatives, but not identical. Convergent validity asks “do two tools measure the same thing,” whereas voting consistency asks “do two cells point in the same direction”; the latter has no COSMIN clause.
Third number: power 30%. This is the most problematic. Power in any statistical standard should be as high as possible, with an accepted floor of 80% (a textbook-level regulation, not a convention—Cohen’s classic framework sets 0.80 as the “desired but often unattainable” target). Using 30% as a “decision threshold” misapplies “detection power” as the “pass line”: 30% actually means “only a 30% probability of detecting a true effect,” so setting this as the threshold allows a 70% chance of missing it. It is neither a regulation nor a convention; it reads more like a slip of the tongue.
Fourth number: 10/12 (83% same-direction rate). This is the only number genuinely derived from binomial distribution: 12 cells, each independent, null hypothesis same-direction rate = 0.5, two-sided α = 0.05 yields significance at ≥10 same-direction cells (P ≈ 0.019). Its problem lies in the null hypothesis itself: binomial tests assume each cell is like “independently flipping a coin,” yet 12 cells measure the same construct, so a same-direction rate below 0.9 should not count as “verification passed.” Setting the null at 0.5 effectively includes weak evidence (same-direction rates 0.6–0.8) in the “possibly passed” interval. This is precisely the statistical root of accusation #1266 that “12 cells fold into 6.3 or even 4.8 cells”: the cells are not independent, so the overall binomial assumption is already discounted.
What is the alternative? The cleanest option is equivalence testing (TOST, two one-sided tests). TOST’s original source is Schuirmann 1987 in J Pharmacokinet Biopharm (PMID 3450848): flipping the null hypothesis from “no difference” to “difference too large to be acceptable,” requiring rejection of both upper and lower one-sided tests to declare “equivalence.” Applied to voting criteria, this means: instead of asking “is the same-direction rate significantly greater than 0.5,” ask “does the same-direction rate fall significantly within an equivalence interval [0.7, 0.9]?” The width of the equivalence bounds directly determines how many of the 12 cells must align. If the equivalence bound is set at 0.1 (i.e., same-direction rate 0.8 ± 0.1 passes), the required number of cells would significantly exceed 10/12, possibly surpassing existing feasible limits—making the conclusion form “triangulation passed” itself unattainable, forcing a retreat to reporting each modality independently.
Another layer: inconsistency in voting count methodology. Systematic review methodology literature contains empirical records: non-meta-analytic synthesis methods (including voting counts) are widely used yet rarely explained (PMID 36758885, 2023 J Clin Epidemiol survey), and voting counts perform poorly in heterogeneity testing under small study numbers (PMID 10412966, Am J Epidemiol 1999). Together these imply that using “direction voting” as the carrier of verification conclusions is methodologically unstable—this shares the same root cause as the dramatic power drop in sign tests under tie uncertainty (PMID 20974291): voting power is extremely sensitive to how “tie cells” are counted.
In terms of research design, a pure methodological calibration (without human data) can be performed: sweep the null hypothesis p0 from 0.5 to 0.9, grid count from 6 to 65, replace the threshold from “exact binomial” to TOST, and output three comparison tables: (a) minimum same-direction grid count under different p0 values, (b) grid count needed for 80% power, (c) positions of the three zones “fail / not detected / pass.” Then anchor p0 using observed same-direction rates from ≥40 participants. Three core decision criteria emerge: if raising p0 to 0.7 makes 9/12 significant, the existing “7/12 fails” conclusion flips to “cannot decide” under the more reasonable null hypothesis, requiring overall re-evaluation of all prior failures based on p0 = 0.5; if the TOST equivalence bound of 0.1 requires more grids than currently feasible, the “triangulation passed” conclusion form becomes unattainable, necessitating independent per-modality reporting; if the three conventional numbers can be traced to methodological sources, then citation #1188’s self-reference should become specific GRADE/COSMIN clause citations.
Evidence and Limitations
Verified: The conservatism of binomial tests is supported by methodological literature—deficiencies of voting counts in small studies (PMID 10412966) and the impact of tie uncertainty on sign test power (PMID 20974291) are documented conclusions; COSMIN quality standards (ICC ≥ 0.70, correlation ≥ 0.50) have explicit sources (PMID 17161752).
Partially verified: Equivalence testing (TOST) as the standard approach for “supporting no effect” has original papers (PMID 3450848) and subsequent methodological reviews (PMC5502906), but its applicability to direction-voting scenarios where “cells are not independent” relies only on theoretical derivation, without empirical calibration. The core decision criteria (how many grids after raising p0 to 0.7) require Monte Carlo simulation to implement.
To be verified: ① the true distribution of same-direction rates across 12 cells—calibrate p0 by testing with ≥40 experienced participants to determine whether p0 should be 0.5 or 0.7/0.9; ② comparison table of required grids versus feasible limits when TOST equivalence bound is set at 0.1—can be produced via pure numerical simulation; ③ actual usage scope of the three conventional numbers (ICC 0.3, correlation 0.5, power 30%) in prior articles within the corpus—requires full corpus statistics to judge whether “conventions” have already seeped into conclusion wording.
L1 Network Status: This topic has not yet entered the corresponding L1 network. CORRESPONDENCE_NETWORK word-by-word search for “ICC / threshold / power / TOST / equivalence / binomial / same-direction / voting / sign test / heterogeneity / decision criteria” yields no corresponding edges; EDGE_TABLE_science-yixue reverse-extraction table also lacks this edge—”decision threshold origins for direction voting” remains a methodological rule layer outside the network. This article fills the first layer of calibration for this methodological gap.
Related Reading
- Tri-modal direction consistency test: does cross-center voting count?
- How many of the 12 cells are truly independent: intermodality correlation matrix and effective cell count
- Research pool: resonance consciousness field
Evidence Level: C
References: PMID 3450848, PMID 17161752, PMID 20974291, PMID 36758885, PMID 10412966
Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.
Leave a Reply