Spatial Epi Public Health GIS

Detecting Distribution Drift in Daily Case Geography

This guide, part of Spatial Data Quality Monitoring, covers the distribution family of checks: detecting that the geography of today’s data differs from recent history for a reason that is not epidemiological.

Problem Context & Constraints

The failures this check exists for share a signature: every record is well-formed, every geometry is valid, every count reconciles, and the output is still wrong. A geocoder reference update shifts a few thousand addresses from street-segment matches to rooftop matches and moves them tens of metres — enough to cross tract boundaries. A laboratory switches its reporting system and begins sending the ordering provider’s address instead of the patient’s. A county’s feed starts truncating the street number, so everything falls back to ZIP centroids.

Each produces a change in the distribution of case geography with no change in the underlying disease. A cluster statistic sees the change and cannot tell it from transmission.

Three constraints make detection hard. Real epidemiological change also shifts the distribution — an outbreak concentrates cases in one area by definition, so a drift detector that halts on concentration would suppress surveillance exactly when it matters. Daily counts are small and noisy, so a naive comparison of proportions fires constantly. And the drift is often in a variable nobody analyses directly — the match-type mix, the distance to the previous geocode of the same address — which is why it goes unnoticed.

The resolution is to monitor the variables that are informative about data quality and uninformative about disease: match-type mix, reporting-source mix, and displacement of recurring addresses between geocoder vintages. Disease does not change how addresses geocode.

Monitor What Disease Cannot Move Four monitored quantities arranged by what moves them. Geographic share of cases by county moves under both a data change and a genuine outbreak, so it cannot distinguish them on its own. Match-type mix moves under a geocoder or feed change but not under an outbreak, because disease does not affect how an address matches. Reporting-source mix moves when a laboratory changes systems but not when transmission changes. Displacement of recurring addresses between geocoder vintages moves only when the reference data changes. The three disease-insensitive indicators are marked as the reliable drift detectors. A drift detector should be blind to epidemiology monitored quantity moved by data change moved by outbreak county share of cases yes yes match-type mix yes no reporting-source mix yes no recurring-address displacement yes no The bold rows can halt; the top row can only warn

Prerequisites

  • python 3.11
  • pandas 2.2.2, numpy 1.26.4, scipy 1.13.1, geopandas 0.14.4
  • Per-record geocode match type and reporting source
  • A stable address key, so recurring addresses can be compared across geocoder vintages
  • At least several months of daily history of each monitored quantity

Step-by-Step Solution

Compute three disease-insensitive indicators and one disease-sensitive one, compare each against a pooled historical baseline with a test that accounts for batch size, and escalate only the disease-insensitive ones.

# Drift detection on case geography with disease-insensitive indicators.
# Pinned: pandas==2.2.2, numpy==1.26.4, scipy==1.13.1, geopandas==0.14.4
import logging

import numpy as np
import pandas as pd
import geopandas as gpd
from scipy.stats import chi2_contingency

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("dq.drift")


def categorical_drift(today: pd.Series, history: pd.Series, name: str,
                      alpha: float = 0.001, min_expected: float = 5.0) -> dict:
    """Chi-square of today's category counts against pooled history.

    Categories with tiny expected counts are merged into 'other' so the test is
    not dominated by a rare category appearing once.
    """
    cats = history.index.union(today.index)
    h = history.reindex(cats).fillna(0)
    t = today.reindex(cats).fillna(0)
    expected = h / h.sum() * t.sum()
    rare = expected < min_expected
    if rare.any():
        h = pd.concat([h[~rare], pd.Series({"_other": h[rare].sum()})])
        t = pd.concat([t[~rare], pd.Series({"_other": t[rare].sum()})])
    chi2, p, dof, _ = chi2_contingency(np.vstack([t.values, h.values]) + 0.5)
    shift = (t / t.sum() - h / h.sum()).sort_values(key=abs, ascending=False)
    out = {"indicator": name, "chi2": float(chi2), "p": float(p), "dof": int(dof),
           "drifted": bool(p < alpha), "largest_shift": shift.head(3).round(4).to_dict(),
           "n_today": int(t.sum())}
    (log.warning if out["drifted"] else log.info)(
        "%-24s p=%.2e drifted=%s largest=%s", name, p, out["drifted"],
        out["largest_shift"])
    return out


def recurring_address_displacement(today: gpd.GeoDataFrame, prior: gpd.GeoDataFrame,
                                   key: str = "addr_key",
                                   tolerance_m: float = 30.0) -> dict:
    """How far did addresses seen before move when geocoded again today?

    Disease cannot move an address. A distribution of displacements that grows
    means the reference data or the geocoder changed.
    """
    m = today[[key, "geometry"]].merge(prior[[key, "geometry"]], on=key,
                                       suffixes=("_t", "_p"))
    if m.empty:
        return {"indicator": "address_displacement", "n_recurring": 0, "drifted": False}
    d = gpd.GeoSeries(m["geometry_t"], crs=today.crs).distance(
        gpd.GeoSeries(m["geometry_p"], crs=today.crs))
    moved = d > tolerance_m
    out = {"indicator": "address_displacement", "n_recurring": int(len(m)),
           "share_moved": float(moved.mean()), "p95_move_m": float(np.percentile(d, 95)),
           "drifted": bool(moved.mean() > 0.02)}
    (log.warning if out["drifted"] else log.info)(
        "address_displacement     recurring=%d moved>%.0fm=%.2f%% p95=%.0fm drifted=%s",
        out["n_recurring"], tolerance_m, 100 * out["share_moved"],
        out["p95_move_m"], out["drifted"])
    return out


def classify_drift(results: list[dict]) -> str:
    """Escalate only on disease-insensitive indicators."""
    insensitive = {"match_type_mix", "reporting_source_mix", "address_displacement"}
    data_drift = [r for r in results if r["indicator"] in insensitive and r["drifted"]]
    geo_shift = [r for r in results if r["indicator"] == "county_share" and r["drifted"]]
    if data_drift:
        verdict = "DATA_DRIFT"
    elif geo_shift:
        verdict = "GEOGRAPHIC_SHIFT_ONLY"   # could be disease; warn, do not halt
    else:
        verdict = "STABLE"
    log.info("drift verdict: %s", verdict)
    return verdict

The classification rule is the point of the whole design. A shift in the county share with no movement in the disease-insensitive indicators is a candidate epidemiological signal and should go to the analysts, not halt the pipeline. A shift in the match-type mix is a data problem regardless of what the county share does, and should halt or at least block publication until someone looks.

The recurring-address check is the most specific instrument available. An address geocoded last month and again today should land in the same place. When the share of recurring addresses that moved rises from its usual fraction of a per cent to several per cent, the geocoder or its reference data changed, and nothing about disease could have produced that.

The Night the Reference Data Changed A time series of the daily share of recurring addresses that moved more than thirty metres when geocoded again. For weeks the share stays well under one per cent, reflecting occasional corrections. On one night it jumps to about eleven per cent, then returns to baseline the following night once the new reference data is the comparison point. The spike coincides with a geocoder reference update that was not announced, and it is the only indicator that pinpointed the date, because case counts and county shares that night looked unremarkable. Nothing else in the run looked unusual that night 2% drift threshold unannounced reference update 11% of recurring addresses moved eight weeks of nightly runs share of recurring addresses that moved more than 30 m

Validation & Edge Cases

Small batches. A Sunday with forty records has little power to detect a mix change and a high chance of a spurious one. Merging rare categories and using a strict significance level handles most of it; alternatively, run the categorical tests on a rolling multi-day window.

Baseline contamination. If the history window contains a previous drift incident, the baseline absorbs it. Exclude known incident periods from the baseline and record the exclusions.

Legitimate change in the reporting-source mix. A new laboratory joining the feed shifts the source mix permanently. That is correct to flag, and once confirmed the baseline must be reset from the change date rather than letting the alert fire for months.

Displacement from genuine address correction. A small fraction of recurring addresses move because the original geocode was wrong and has been fixed. That is the baseline level and should be quiet; the check measures departures from it.

Drift that arrives gradually. A feed that degrades by a percentage point a week never trips a test against a rolling baseline, because the baseline moves with it. Keep one comparison against a fixed reference period alongside the rolling one; the fixed comparison catches slow degradation and the rolling one catches abrupt changes, and neither does both.

Drift confined to one jurisdiction. A state-level test of the match-type mix can pass while one county’s feed has collapsed to ZIP centroids, because that county is a small share of the total. Run the categorical tests per reporting jurisdiction as well as pooled, correcting for the number of jurisdictions tested so the per-county checks do not produce a false alarm most nights.

Coordinate reference system mismatch between runs. Comparing today’s projected coordinates with a prior run’s geographic coordinates produces enormous “displacements” for every address. Assert both are in the working system before measuring distance.

WARNING dq.drift match_type_mix           p=3.14e-21 drifted=True largest={'street_interpolated': -0.1822, 'rooftop': 0.1706}
WARNING dq.drift address_displacement     recurring=842 moved>30m=11.16% p95=64m drifted=True
INFO dq.drift county_share             p=4.02e-01 drifted=False largest={'06075': 0.0121}
INFO dq.drift drift verdict: DATA_DRIFT

The per-jurisdiction test in that list needs a multiple-comparison correction, and the arithmetic of testing many counties every night explains why:

Many Counties, Many Chances for a False Alarm Expected number of false drift alarms per night against the number of jurisdictions tested separately. Without correction, testing each county at a fixed significance level produces a false alarm count that grows linearly with the number of counties, reaching several per night for a large state. With a correction that controls the family-wise or false discovery rate, the expected number stays near zero regardless of how many counties are tested, so per-county drift checks remain usable. Per-county checks are only usable with a correction uncorrected corrected 1 50 100 number of jurisdictions tested each night false alarms per night

Compliance Notes

Record, per run, each indicator with its test statistic, p-value and largest category shifts; the baseline window and any excluded incident periods; the displacement tolerance and the share of recurring addresses that moved; and the drift verdict with the action taken.

A data drift that reaches publication has a disclosure dimension as well as an accuracy one: if a geocoder change moved cases across a suppression boundary, a cell that was safely above the minimum may now be below it, or the reverse. Where a drift verdict is raised, the suppression stage should be re-run against the corrected data before anything is released, and the incident belongs in the product’s lineage record.