三千年总集漏记了多少?用「双清单捕获-再捕获」给史书算个账

Written by

in

Language

Evidence level: C

古人记灾异,像渔夫撒网——网住了多少鱼看得见,漏掉了多少鱼,得用统计学才能算出来。

正文

《中国三千年气象记录总集》这类文献汇编,把正史、方志、笔记里的旱、涝、霜、雪、蝗灾一条条抄下来,是研究历史气候与「灾异」观念的第一手材料。但一个绕不开的问题是:这份清单到底漏了多少? 战乱年代没人记、偏远省份没人抄、小事不上史官的笔——漏记率不是均匀的,它本身就是一个值得测量的量。

生态学里有个老办法叫捕获-再捕获(capture-recapture):渔夫第一网捞住 n₁ 条鱼做标记放回,第二网捞 n₂ 条,其中有 m 条带标记,那湖里总鱼数约等于 n₁×n₂/m。流行病学界把这套方法借来估计疾病登记系统的漏报率:拿两份来源独立的病例清单,数一数重叠的部分,就能反推「两份清单都没碰到的」那些病例有多少。

套到三千年总集上,设计是这样的:清单 A = 文献事件(某年某省旱,出自某方志);清单 B = 代用记录转译出的独立事件(树轮宽度极低年、石笋 δ¹⁸O 异常段、冰芯事件层,按「时间×地点×类型」模糊匹配转成事件)。两清单的重叠数 m 就是关键——重叠越多,说明两边网眼都密,漏记率低;重叠少,湖里藏的鱼就多。

这个思路并非空想。树轮学家早就拿历史文献给树轮序列「对账」:甘肃葫芦河流域的树轮降水重建,把 1691 年以来的大旱年与历史档案逐条对照,多数极端旱年能对上(PMID 21701959);黄土高原西部 283 年的树轮干旱指数,也与方志整理出的旱涝等级序列同步起伏(PMID 26207621)。能对上,说明两清单测的是同一批「真事件」——这正是捕获-再捕获成立的前提。

但直接套公式会踩三个坑,每一个都有流行病学实证教训:

坑一:独立性假设。 经典两清单法要求两份清单彼此独立。如果文献记录和树轮都偏向记录「大旱」而放过小旱,两清单呈负相关,捕获-再捕获会严重高估总事件数(PMID 7888444 的癌症登记模拟明确演示了这一点)。缓解办法是按事件强度分层,层内再估。

坑二:匹配错误。 「某年山西旱」对上树轮低值年,靠的是时间×地点×类型的模糊匹配,配错了会虚增重叠数。Seber 等人用「标签丢失」模型把清单录入错误率一起估出来,发现错误率高时忽略它会导致总数被大幅高估(PMID 11129484)。

坑三:异质性。 儿童车祸伤害数据的双清单分析显示,匹配概率随年龄、事故类型显著变化,估计出来的总数可在 1729 到 2743 之间摆动(PMID 10728542)——所以「分省、分事件类型、分朝代」不是锦上添花,而是必须。这正是本课题要求产出分省漏记率矩阵的原因:它同时是一张史官制度与方志传统的「文化地理画像」。

落到实操,还有两道工序值得交代。第一道是「事件化」:树轮宽度、石笋 δ¹⁸O 本身是连续序列,不是事件清单,得先定阈值——比如树轮低值超过均值减 1.5 个标准差算一次「捕获」,石笋异常段按年代学误差给事件一个年份区间。阈值取多严,直接决定清单 B 的网眼密度,所以敏感性分析要横跨多个阈值重跑。第二道是估计器的选择:重叠数偏小时的经典公式方差极大,小样本情形下应改用 Chao 下界估计器,它利用「只见过一次的物种数」修正偏差——这在比利时沙门氏菌监测覆盖评估等现代登记系统中已是标准做法,思路可直接迁移。

对易学研究而言,这项工作的意义在于把「史书里的灾异」从「天人感应的文本证据」还原成「带已知偏差的观测数据」。漏记率矩阵一旦建立,任何「某朝灾异特别多所以天命有变」式的论证,都要先过「该省该代漏了多少」这一关。

证据与局限

已验证:捕获-再捕获在医学登记完整性评估中是成熟方法;树轮/代用记录与历史文献能逐条对账(PMID 21701959、26207621)。

部分验证:方法各组件分别成立,但「文献×代用记录」这一特定组合的双清单估计尚无完整实例——代用记录需先「事件化」(阈值转事件),其捕获概率结构不同于病例登记。

待验证:分省漏记率矩阵的可估计性(某些省代用记录覆盖为零,无法闭环);两清单依赖方向的经验判断;模糊匹配错误率的实际量级。该主题尚未进入 L1 对应网络,本文证据等级封顶 C。

相关阅读

证据等级: C
参考文献: PMID 11129484, PMID 7888444, PMID 10728542, PMID 21701959, PMID 26207621

L1 网络状态:该主题尚未进入 L1 对应网络(CORRESPONDENCE_NETWORK.md 与 EDGE_TABLE_science-yixue.md 中无「总集漏记率 / 捕获-再捕获 / 代用记录对账」相关边)。


免责说明:本文属传统知识与科学文献的整理、对照与研究设计讨论,仅供理解与参考,不构成医疗建议,不能替代专业诊断与治疗。文中涉及的效应量、频率区间与暴露结论多为研究报告层面的推断,个体情况差异很大;如有身体不适或健康疑虑,请就医并由专业医师判断。

How much was omitted in the three-thousand-year compilation? Balancing the historical records with a \”double-list capture-recapture\” method.

Evidence level: C

Ancient records of disasters and anomalies resemble a fisherman casting a net—the number of fish caught is visible, but the quantity of fish missed can only be estimated through statistical methods.

Body

Compilations such as The Collected Records of China’s Three Millennia aggregate primary materials on historical climate and the concept of “disasters and anomalies” by transcribing droughts, floods, frosts, snows, and locust plagues from official histories, local gazetteers, and notes. However, an unavoidable question arises: How many events does this list omit? During wars, no one recorded; remote provinces were ignored; minor incidents never reached the historian’s brush—the omission rate is not uniform; it itself constitutes a measurable quantity.

Ecology employs an established method called capture-recapture: fishermen cast the first net, catch $n_1$ fish, mark them, and release them; the second net catches $n_2$, among which $m$ are marked, so the total fish in the lake is approximately $n_1 \times n_2 / m$. The epidemiology community adapted this method to estimate underreporting in disease registries: by comparing two independent case lists and counting the overlap, one can infer the number of cases missed by both lists.

Applied to the three-millennia compilation, the design is as follows: List A = documented events (e.g., “Drought in Province X in Year Y” from Gazetteer Z); List B = independently translated events derived from proxy records (tree-ring width minima, speleothem $\delta^{18}$O anomalies, ice-core event layers), matched via “time × location × type” into discrete events. The overlap count $m$ is critical—the more overlap, the denser both nets, implying lower omission; less overlap suggests more hidden events.

This approach is not hypothetical. Dendrochronologists have long cross-checked historical texts against tree-ring sequences: reconstructed precipitation in the Hulu River basin, Gansu, aligns with historical archives for major droughts since 1691 (PMID 21701959); the 283-year tree-ring drought index in western Loess Plateau synchronizes with the flood-drought sequence compiled from gazetteers (PMID 26207621). When they align, both lists measure the same set of “true events”—the prerequisite for capture-recapture.

Yet directly applying the formula risks three pitfalls, each illustrated by epidemiological evidence:

Pitfall 1: Independence assumption. Classic two-list methods require mutual independence. If both literature and tree rings preferentially record “severe droughts” while ignoring mild ones, the lists become negatively correlated, causing capture-recapture to grossly overestimate total events (PMID 7888444’s cancer registry simulation explicitly demonstrates this). Mitigation involves stratifying by event intensity and estimating within strata.

Pitfall 2: Matching error. Matching “Drought in Shanxi, Year X” to a tree-ring low year relies on fuzzy matching of time × location × type; mismatches inflate overlap artificially. Seber et al. used a “tag-loss” model to jointly estimate list-entry error rates, finding that ignoring high error rates leads to substantial overestimation of totals (PMID 11129484).

Pitfall 3: Heterogeneity. Two-list analysis of child car accident injury data shows matching probability varies significantly with age and accident type, causing estimated totals to swing between 1729 and 2743 (PMID 10728542)—thus stratification by province, event type, and dynasty is not optional but necessary. This explains why this project requires producing a province-specific omission rate matrix: it simultaneously serves as a cultural-geographic portrait of the official historiography and local gazetteer traditions.

Operationally, two additional steps merit attention. First is “eventization”: tree-ring width and speleothem $\delta^{18}$O are continuous sequences, not event lists; thresholds must be set—for example, tree-ring values below mean minus 1.5 standard deviations count as one “capture,” while speleothem anomalies are assigned year ranges based on dating uncertainty. How strict the threshold is directly determines List B’s mesh density, necessitating sensitivity analysis across multiple thresholds. Second is estimator selection: when overlap counts are small, classical formulas yield enormous variance; for small samples, the Chao lower-bound estimator should be used, correcting bias via “species seen only once”—a standard practice in modern registries such as Belgian Salmonella surveillance coverage assessment, whose logic can be directly transferred.

For Yijing Studies, the significance of this work lies in transforming “disasters and anomalies in historical texts” from “textual evidence of heaven-man resonance” into “observed data with known biases.” Once the omission rate matrix is established, any argument like “Dynasty X had many disasters, so the Mandate of Heaven changed” must first pass the hurdle: “How many events were omitted in Province Y of Dynasty X?”

Evidence and Limitations

Verified: Capture-recapture is a mature method for assessing medical registry completeness; tree-ring/proxy records can be reconciled item-by-item with historical documents (PMID 21701959, 26207621).

Partially Verified: While individual components of the methodology hold, a complete dual-list estimate for the specific combination of “documents × proxy records” remains absent—proxy records must first be “evented” (thresholds converted to events), and their capture probability structure differs from case registries.

Pending Verification: The estimability of the provincial omission rate matrix (certain provinces have zero proxy record coverage, preventing closure); empirical judgment of the directionality dependence between the two lists; and the actual magnitude of fuzzy-matching error rates. This topic has not yet entered the L1 corresponding network; consequently, the evidence level for this paper is capped at C.

Related Reading

Evidence Level: C
References: PMID 11129484, PMID 7888444, PMID 10728542, PMID 21701959, PMID 26207621

L1 Network Status: This topic has not yet entered the L1 correspondence network (no edges related to “aggregate omission rate / capture-recapture / proxy record reconciliation” are present in CORRESPONDENCE_NETWORK.md and EDGE_TABLE_science-yixue.md).


Disclaimer: This article compiles traditional knowledge and scientific literature for comparison and research-design discussion only. It is not medical advice and cannot replace professional diagnosis or treatment. Effect sizes, frequency ranges and exposure conclusions cited here are largely inferences at the research-report level, and individual circumstances vary widely. If you have any discomfort or health concerns, please consult a physician.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *