Spatial Epi Public Health GIS

Measuring Utility Loss After Geomasking

A geomask is chosen for its privacy guarantee and lived with for its analytic cost, and the second is usually estimated by intuition. This guide, part of Geographic Masking Techniques, sets out a measurement procedure that produces a number for each analysis a released file is meant to support.

Problem Context & Constraints

Utility loss is not a single quantity. The same mask that leaves a county rate map essentially untouched can destroy a short-distance point-pattern estimate, and reporting “median displacement 500 m” tells a downstream analyst nothing about whether their specific question survives.

The measurement is straightforward in principle: run the intended analysis on the true data and on the masked data, and compare. Two constraints complicate it.

The comparison must happen inside the trusted environment, because it requires the true locations. That means utility measurement is a data-holder activity, and its output — the utility report — is what travels with the released file rather than the comparison itself.

And the mask is stochastic, so a single realisation is not the answer. Every mask draw produces a different displaced file and a different utility outcome, so the measurement is a distribution over draws, and reporting a single run understates the variability the analyst will experience.

One Mask, Five Different Costs Utility retained across three donut-mask radii for five analyses. County rate mapping retains essentially all utility at every radius. Tract rate mapping falls from 98 to 88 percent. Kernel density at a two-kilometre bandwidth falls from 96 to 74. Nearest-facility assignment falls from 93 to 61. Ripley's K below 500 metres falls from 71 to 14, and is unusable at the largest radius. The mask that is harmless for four analyses destroys the fifth. The same mask, harmless four times and fatal once 250 m 500 m 1000 m County rates 100% 100% 99% Tract rates 98% 94% 88% KDE, 2 km 96% 87% 74% Nearest facility 93% 79% 61% Ripley's K < 500 m 71% 38% 14% Publish this table with the file, and the analyst knows which questions it can answer

Prerequisites

  • The true point locations, inside the trusted environment
  • A masking implementation with a settable radius and a seed
  • A written list of the analyses the released file is meant to support
  • python 3.11, geopandas 1.0.1, numpy 1.26.4, pandas 2.2.2, esda 2.6.0

Utility measurement is a data-holder activity, and being explicit about what crosses the boundary is part of the design:

What Stays Inside and What Is Released A trusted environment containing the true locations, the mask realisations and the per-analysis comparisons. Only two things leave it: the masked point file itself, and a summary utility table naming the analyses tested and the retention achieved. The comparisons that produced the table require the true locations and therefore never leave, which is why the utility report is a deliverable rather than something a recipient can reproduce. Two things leave; everything that produced them stays trusted environment true locations 20 mask realisations per-analysis comparisons none of this is reproducible outside masked point file one realisation, with its seed utility table analyses tested and retention The recipient cannot compute the second, which is exactly why it has to be supplied

Step-by-Step Solution

# Measure utility retention across mask draws, per analysis.
# Pinned: geopandas==1.0.1, numpy==1.26.4, pandas==2.2.2, scipy==1.13.1
import logging
import numpy as np
import pandas as pd
import geopandas as gpd
from scipy.stats import spearmanr

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("mask.utility")


def areal_utility(truth: gpd.GeoDataFrame, masked: gpd.GeoDataFrame,
                  areas: gpd.GeoDataFrame, id_col: str = "geoid") -> float:
    """Spearman correlation of per-area counts. 1.0 = the map is unchanged."""
    def counts(pts):
        j = gpd.sjoin(pts, areas[[id_col, "geometry"]], how="inner", predicate="within")
        return j.groupby(id_col).size()
    a, b = counts(truth), counts(masked)
    idx = a.index.union(b.index)
    rho, _ = spearmanr(a.reindex(idx, fill_value=0), b.reindex(idx, fill_value=0))
    return float(rho)


def assignment_utility(truth: gpd.GeoDataFrame, masked: gpd.GeoDataFrame,
                       facilities: gpd.GeoDataFrame) -> float:
    """Share of cases assigned to the same nearest facility before and after."""
    t = gpd.sjoin_nearest(truth, facilities[["fac_id", "geometry"]], how="left")["fac_id"]
    m = gpd.sjoin_nearest(masked, facilities[["fac_id", "geometry"]], how="left")["fac_id"]
    return float((t.to_numpy() == m.to_numpy()).mean())


def utility_over_draws(truth: gpd.GeoDataFrame, mask_fn, analyses: dict,
                       radii, n_draws: int = 20, seed: int = 42) -> pd.DataFrame:
    """Utility per analysis per radius, across independent mask realisations.

    Reporting the mean across draws AND the spread matters: an analyst receives
    one realisation, not the mean, so the lower quantile is what they should
    plan against."""
    rng = np.random.default_rng(seed)
    rows = []
    for r in radii:
        for d in range(n_draws):
            masked = mask_fn(truth, radius=r, seed=int(rng.integers(1 << 31)))
            for name, fn in analyses.items():
                rows.append({"radius": r, "draw": d, "analysis": name,
                             "utility": float(fn(truth, masked))})
    df = pd.DataFrame(rows)
    summ = (df.groupby(["analysis", "radius"])["utility"]
              .agg(mean="mean", p10=lambda s: float(np.percentile(s, 10))).reset_index())
    log.info("utility summary:\n%s", summ.to_string(index=False))
    return summ


def choose_radius(summary: pd.DataFrame, requirements: dict) -> dict:
    """Largest radius meeting every stated utility requirement.

    Larger radius means stronger privacy, so the correct choice is the largest
    that still satisfies the analyses the file must support."""
    ok = None
    for r in sorted(summary["radius"].unique()):
        at_r = summary.loc[summary["radius"] == r].set_index("analysis")["p10"]
        if all(at_r.get(a, 0.0) >= need for a, need in requirements.items()):
            ok = r
    if ok is None:
        raise ValueError("no radius in the sweep meets every requirement; relax a "
                         "requirement or release an areal product instead")
    log.info("selected radius %g m", ok)
    return {"radius": ok}

Choosing the largest radius that still meets the requirements inverts the usual habit. Displacement is the privacy protection, so the goal is to give away as much location precision as the analyses can tolerate, not as little as the analyst will accept.

Validation & Edge Cases

1. Use the tenth percentile across draws, not the mean. The analyst receives one realisation. Planning against the mean means half of released files perform worse than the stated utility:

INFO utility summary:
        analysis  radius  mean   p10
     county_rate     250 0.998 0.996
     county_rate    1000 0.991 0.986
      tract_rate     250 0.981 0.972
      tract_rate    1000 0.884 0.851
nearest_facility    1000 0.612 0.579
INFO selected radius 500 m

2. Include an analysis you expect to fail. A utility table where everything passes is not informative about where the limit is. Including a short-distance point-pattern estimate shows the boundary and pre-empts a user attempting it.

3. Re-measure per release, not once. Utility depends on the point density and the geography, so a radius validated on an urban county is not validated for a rural one, and the same programme may need different radii in different areas — which is itself an argument for the density-adaptive masking described in the parent guide.

4. Do not let the analyst choose the radius. The utility table is theirs to read; the radius is a disclosure-control decision belonging to the data holder, informed by the spatial k-anonymity validation rather than by analytic convenience.

5. Publish the table even when it is unflattering. A file that supports county rates and nothing else is useful if that is stated, and actively harmful if the recipient assumes otherwise.

6. Measure utility on the population that matters, not the average. A mask that preserves 90% of utility overall can preserve 98% in dense areas and 40% in sparse ones, and the sparse areas are usually the ones an equity analysis is about. Break the utility scores down by the same strata used for the privacy validation, so the two halves of the trade-off are reported on the same partition of the map.

7. Include the analyses the recipient will actually run, not the ones you would run. Ask. A research partner planning a case-control study with distance-to-source exposure has a very different utility requirement from a programme office producing a county rate table, and a utility report built around the wrong analyses is reassuring about the wrong thing.

8. Re-measure when the facility layer changes. Nearest-facility utility depends on where the facilities are, so a clinic opening or closing changes the assignment agreement without any change to the mask. Where a released file supports assignment analyses, its utility statement is tied to the facility layer that was current when it was measured, and that layer’s vintage belongs in the report.

9. Treat the utility table as part of the release, not as documentation. It has a version, it is tied to a specific mask seed and facility layer, and it becomes wrong when either changes. Giving it the same version identifier as the data file, and updating both together, keeps the pair coherent in a way that a separate methods note reliably does not.

Reporting the mean across mask draws flatters the file the recipient actually gets, and the gap is worth showing:

Mean Utility Against the Tenth Percentile Utility retained for four analyses at a five hundred metre radius, reported as the mean across twenty mask draws and as the tenth percentile. County rates are 100 percent at the mean and 100 at the tenth percentile. Tract rates are 94 and 89. Kernel density is 87 and 79. Nearest-facility assignment is 79 and 68. The gap widens as the analysis becomes more sensitive, and the recipient receives one draw rather than the mean. The recipient gets one draw, not the mean mean 10th percentile County rates 100 Tract rates 94 89 Nearest facility 79 68

Compliance Notes

  • The utility measurement runs on true data and its intermediate outputs are as sensitive as the source file. Only the summary table leaves the trusted environment.
  • Record the mask seed with the release, since a re-run must reproduce the released file exactly for any later audit.
  • Publish the utility table alongside the file, naming the analyses tested and the retention achieved, so the recipient does not have to guess.
  • Re-run the spatial k-anonymity validation whenever the radius changes, because utility and privacy move in opposite directions and only one of them is being optimised here.