Spatial Epi Public Health GIS

Density-Adaptive Geomasking at Scale

This guide, part of Geographic Masking Techniques for Point-Level Health Data, covers masking large case files — a year of statewide reportable disease records — with displacement tuned to where each case lives.

Problem Context & Constraints

A fixed donut mask displaces every case by, say, 100 to 500 metres. In a dense city, 500 metres contains thousands of households, so the protection is far more than needed and the displacement blurs detail for nothing. In a rural county, 500 metres may contain three farmhouses, so the protection is far less than needed. One distance cannot serve both.

Density-adaptive masking sets each case’s displacement from the population around it. For a target of k households, the outer radius is the distance to the kth-nearest household, and the inner radius a fraction of that. Urban cases move a few dozen metres; rural cases move kilometres. Every case gets roughly the same protection, measured in households, and no more displacement than that protection requires.

Scale brings its own constraints. A statewide file may hold several hundred thousand cases and several million address points. Performance: nearest-neighbour queries must be vectorised against a spatial index built once, not computed per case. Reproducibility: the mask must be random, yet the same input must produce the same output when rerun, or a correction to one record would re-mask everything and let two versions be averaged. Plausibility: masked points should land where people live, not in lakes or industrial parks, or users will spot and discard them. Verification: the achieved k must be measured on the output, not assumed.

Same Protection, Different Distances Two rows compare masks. With a fixed five-hundred-metre donut, an urban case is hidden among about two thousand households, far more than needed, while a rural case is hidden among only three. With a density-adaptive donut targeting twenty households, the urban donut shrinks to about sixty metres and the rural donut grows to about two kilometres, and both cases are hidden among roughly twenty households. Displacement follows density; protection stays level Fixed 500 m urban: ~2,000 households rural: ~3 households Adaptive k = 20 urban: ~60 m, 20 households rural: ~2 km, 20 households circle sizes are schematic, not to scale

Prerequisites

  • python 3.11
  • numpy 1.26.4, pandas 2.2.2, geopandas 0.14.4, scipy 1.13.1, pyarrow 16.1.0
  • Case points in a projected metric coordinate reference system, with a stable record identifier
  • Residential address points (or a household-weighted point layer from census blocks) for the same area and a few kilometres beyond it
  • A secret masking key held outside the code repository, for deterministic per-record randomness
  • The target k agreed for the release, as in Choosing K for Rare Disease Point Releases

Step-by-Step Solution

Build a KD-tree over household points once, compute each case’s kth-neighbour distance in vectorised batches, draw displacement from per-record deterministic randomness, snap each masked point to a nearby residential point so it lands plausibly, and verify the achieved k.

# Density-adaptive donut geomasking for large case files.
# Pinned: numpy==1.26.4, pandas==2.2.2, geopandas==0.14.4, scipy==1.13.1, pyarrow==16.1.0
import hashlib
import hmac
import logging
import os

import numpy as np
import pandas as pd
from scipy.spatial import cKDTree

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("adaptive_mask")

K = 20
INNER = 0.5            # inner radius as a fraction of the kth-neighbour distance
BATCH = 50_000
SNAP_CANDIDATES = 8    # land on one of the nearest residential points to the draw


def record_uniforms(ids: pd.Series, key: bytes) -> np.ndarray:
    """Three uniforms per record from HMAC(key, id): same record, same draw, every run."""
    out = np.empty((len(ids), 3))
    for n, rid in enumerate(ids.astype(str)):
        h = hmac.new(key, rid.encode(), hashlib.sha256).digest()
        a, b, c = (int.from_bytes(h[i:i + 8], "big") / 2**64 for i in (0, 8, 16))
        out[n] = (a, b, c)
    return out


def mask(cases: pd.DataFrame, homes_xy: np.ndarray, key: bytes) -> pd.DataFrame:
    tree = cKDTree(homes_xy)
    out = []
    for start in range(0, len(cases), BATCH):
        c = cases.iloc[start:start + BATCH]
        xy = c[["x", "y"]].to_numpy()
        r_k = tree.query(xy, k=K, workers=-1)[0][:, -1]
        u = record_uniforms(c["record_id"], key)
        theta = 2 * np.pi * u[:, 0]
        r = np.sqrt((INNER * r_k) ** 2 + u[:, 1] * (r_k ** 2 - (INNER * r_k) ** 2))
        draw = xy + np.column_stack([r * np.cos(theta), r * np.sin(theta)])

        # Snap to a nearby residential point so masked cases land where people live,
        # choosing among a few candidates to avoid always picking the same one.
        _, cand = tree.query(draw, k=SNAP_CANDIDATES, workers=-1)
        pick = cand[np.arange(len(c)), (u[:, 2] * SNAP_CANDIDATES).astype(int)]
        masked = homes_xy[pick]
        out.append(pd.DataFrame({"record_id": c["record_id"].to_numpy(),
                                 "mx": masked[:, 0], "my": masked[:, 1],
                                 "r_k": r_k, "disp": np.hypot(*(masked - xy).T)}))
        log.info("masked %d/%d", min(start + BATCH, len(cases)), len(cases))
    return pd.concat(out, ignore_index=True)


def achieved_k(cases: pd.DataFrame, masked: pd.DataFrame, homes_xy: np.ndarray) -> np.ndarray:
    """Households closer to the masked point than the true point is: the attacker's
    candidate set if they search outward from the released location."""
    tree = cKDTree(homes_xy)
    true_xy = cases[["x", "y"]].to_numpy()
    m_xy = masked[["mx", "my"]].to_numpy()
    d_true = np.hypot(*(true_xy - m_xy).T)
    return np.array([len(tree.query_ball_point(p, d)) for p, d in zip(m_xy, d_true)])


def main(cases, homes_xy):
    key = os.environ["MASK_KEY"].encode()
    m = mask(cases, homes_xy, key)
    k_ach = achieved_k(cases, m, homes_xy)
    log.info("achieved k: min=%d p05=%d median=%d; below target=%.2f%%",
             k_ach.min(), np.percentile(k_ach, 5), np.median(k_ach), 100 * np.mean(k_ach < K))
    return m.assign(achieved_k=k_ach)

Deterministic randomness is the key to safe reruns. Each record’s displacement comes from an HMAC of its identifier under a secret key, so the same record always masks to the same location. Correcting one record’s address re-masks only that record, and an attacker cannot collect several independently masked versions of the same case to average. The key must be kept secret: anyone with it and the true address could reproduce the mask, and anyone with it and the masked point could narrow the search.

Snapping to residential points keeps masked cases plausible, which matters for users and for protection: a point in the middle of a lake is obviously displaced and tells an attacker the true location is on the shore. Choosing among several nearby candidates, rather than the single nearest, stops the snap from concentrating many masked cases on the same address.

Displacement Where It Is Needed Bars show median displacement for a statewide case file masked at k equals twenty. Urban core cases moved a median of fifty-five metres, suburban cases one hundred and forty metres, small-town cases four hundred and ten metres, and rural cases one thousand nine hundred metres. Achieved k was at or above twenty for ninety-seven percent of cases in every setting. Median displacement at k = 20 urban core 55 m suburban 140 m small town 410 m rural 1,900 m achieved k at or above target for 97% of cases in every setting

Validation & Edge Cases

Achieved k below target. Snapping and the inner radius can leave some cases with fewer than k households between the masked and true point. Re-mask those records with a larger radius multiplier, keyed on a derived identifier so the result stays deterministic, until every record meets the target or is flagged for withholding.

Study-area edges. Cases near a state line or coastline have fewer households on one side, so the kth-neighbour distance is long, and draws can land outside the study area. Include address points a few kilometres beyond the boundary, and reject draws that fall outside the permitted area.

Group quarters. Nursing homes, dormitories and prisons put many residents at one address point. Count residents, not address points, for those locations, or they will appear as one household and inflate displacement around them.

Clusters of cases. When several cases live close together — a household outbreak — their masks are independent, and the masked points may scatter widely. That protects them, but it removes the cluster. If the release supports cluster analysis, measure how well such clusters survive with the utility checks in Measuring Utility Loss After Geomasking.

Key management. Rotating the key re-masks everything, which creates a second independent version of every record. Rotate only when the key is compromised, and treat the old and new masked files as a combined release for disclosure purposes.

Performance. The HMAC loop in pure Python processes a few hundred thousand records in seconds, which is adequate. The achieved-k check with ball queries is slower in dense areas; sample it for very large files and run it in full only for records in low-density areas, where shortfalls occur.

Address layer currency. Household points from an old address file miss new subdivisions, so cases in new housing appear to live in sparse areas and are displaced further than they need to be; demolished housing does the opposite. Refresh the address layer each year and record its date with the masked file, since the achieved protection depends on it as much as on k.

INFO adaptive_mask masked 412880/412880
INFO adaptive_mask achieved k: min=9 p05=21 median=34; below target=2.71%
INFO adaptive_mask re-mask pass 1: below target=0.08%; withheld after pass 2: 41 records
Mask, Measure, Re-mask Four stages. Mask all records with the density-adaptive donut. Measure achieved k for every record. Re-mask records below the target with a larger radius, keyed deterministically. Records still below target after two passes are withheld from the point release and reported only in aggregate. The final file records achieved k per record for the residual risk statement. Protection is measured on the output, not assumed Mask all adaptive donut Measure k every record Re-mask short larger radius Withhold rest aggregate only achieved k per record goes into the residual risk statement

Compliance Notes

Record the target k, inner radius fraction, household point source and date, snapping rule, key identifier (never the key), the achieved k distribution, re-mask passes, withheld record counts, and displacement by urbanicity. Keep the true-to-masked mapping inside the secure environment and destroy intermediate files that contain both.

The masking key is as sensitive as the case addresses themselves. Store it in the agency’s secrets manager with access limited to the masking service, log every use, and document who can retrieve it. A masked file released under a key later exposed should be treated as unmasked for disclosure purposes.