Spatial Epi Public Health GIS

Handling Missing Geocodes in Exposure Assignment

This guide, part of Assigning Exposure to Residential Locations, deals with the fraction of every surveillance file that has no usable coordinate — typically 3% to 15% — and the three defensible things you can do about it.

Problem Context & Constraints

Geocoding fails for structural reasons, not random ones. Rural route addresses have no street number to match. Post office boxes point to a facility. New construction is absent from an address reference with an older vintage. Multi-unit buildings match the building, not the unit, which is usually fine for exposure and not for anything else. Non-standard address formats, common in some communities and among recent arrivals, fail at higher rates.

Every one of those failure modes correlates with something a health study cares about — rurality, housing tenure, income, recency of arrival. So the records without a coordinate differ systematically from the records with one, and the default treatment of dropping them is a non-random exclusion applied silently.

The size of the problem is easy to underestimate because it compounds. A 92% geocoding success rate looks healthy. Combined with a requirement for a rooftop or parcel match, it may leave 70% of records assignable, and if the analysis also requires a residential history it may leave 45%. Each filter is individually reasonable; together they define a study population that no longer resembles the one the surveillance system covers.

Three options exist, and the choice must be justified rather than defaulted to. Complete-case exclusion is defensible only if you can show the missingness is unrelated to the outcome. Coarse assignment uses the best geography available — a ZIP code, a county — and accepts an area-level exposure for those records. Multiple imputation samples plausible locations within the known containing geography and propagates the uncertainty into the estimate.

Three Strategies and the Assumption Each One Makes Three options laid out side by side for records with no usable geocode. Complete-case exclusion drops them, assumes the missingness is unrelated to both exposure and outcome, and costs external validity when that assumption fails. Coarse assignment gives them an area-level exposure from the containing ZIP code or county, assumes the area mean is a reasonable proxy, and costs contrast because everyone in the area receives the same value. Multiple imputation samples plausible locations inside the known containing geography and pools across draws, assumes the within-area population distribution is known, and costs implementation effort while being the only option that propagates the uncertainty. Every option makes an assumption — state which one you chose Exclude assumes: missing at random with respect to the outcome costs: external validity defensible only with evidence, never as a default the most common choice Coarse assignment assumes: the area mean is a reasonable proxy costs: exposure contrast everyone in the area gets the same number keeps the records in the study Multiple imputation assumes: the within-area population surface is known costs: implementation effort the only option that carries the uncertainty forward widens the interval honestly Run at least two and report both — the difference is the sensitivity to the choice If exclusion and imputation disagree materially, the missingness is doing the work

Prerequisites

  • python 3.11
  • numpy 1.26.4, pandas 2.2.2, geopandas 0.14.4
  • Case records with geocode match type and, for unmatched records, the coarsest known geography — a ZIP code, tract or county
  • A population raster or block-level population for the containing geography
  • An exposure surface covering the whole study area

Step-by-Step Solution

Start by diagnosing the missingness pattern, then implement the strategy chosen from that diagnosis, and report the alternatives alongside.

# Diagnose geocode missingness and impute exposure for unassignable records.
# Pinned: numpy==1.26.4, pandas==2.2.2, geopandas==0.14.4
import logging

import numpy as np
import pandas as pd
import geopandas as gpd

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("exposure.missing")

SEED = 20260918


def missingness_profile(cases: pd.DataFrame, assignable_col: str,
                        covariates: list[str]) -> pd.DataFrame:
    """Compare assignable and unassignable records on every covariate available."""
    rows = []
    a = cases[cases[assignable_col]]
    u = cases[~cases[assignable_col]]
    for c in covariates:
        if pd.api.types.is_numeric_dtype(cases[c]):
            rows.append({"covariate": c, "assignable": float(a[c].mean()),
                         "unassignable": float(u[c].mean()),
                         "kind": "mean"})
        else:
            top = cases[c].mode().iat[0]
            rows.append({"covariate": c, "assignable": float((a[c] == top).mean()),
                         "unassignable": float((u[c] == top).mean()),
                         "kind": f"share_{top}"})
    df = pd.DataFrame(rows)
    df["difference"] = df["unassignable"] - df["assignable"]
    log.info("missingness: %d/%d unassignable (%.1f%%); largest covariate gap: %s",
             len(u), len(cases), 100.0 * len(u) / len(cases),
             df.reindex(df["difference"].abs().sort_values(ascending=False).index)
               .iloc[0]["covariate"])
    return df


def impute_exposure(unassignable: pd.DataFrame, areas: gpd.GeoDataFrame,
                    pop_points: gpd.GeoDataFrame, exposure_at: callable,
                    n_draws: int = 20) -> pd.DataFrame:
    """Sample plausible residences inside the known area, weighted by population.

    unassignable: study_id, area_id (the coarsest geography actually known)
    pop_points:   candidate residential locations with a 'weight' column and area_id
    exposure_at:  callable(GeoDataFrame) -> array of exposure values
    """
    rng = np.random.default_rng(SEED)
    by_area = {a: g for a, g in pop_points.groupby("area_id")}
    draws = []
    for d in range(n_draws):
        picks = []
        for rec in unassignable.sort_values("study_id").itertuples():
            pool = by_area.get(rec.area_id)
            if pool is None or len(pool) == 0:
                picks.append(None)
                continue
            w = pool["weight"].to_numpy(float)
            w = w / w.sum()
            picks.append(pool.iloc[rng.choice(len(pool), p=w)].geometry)
        gdf = gpd.GeoDataFrame({"study_id": unassignable.sort_values("study_id")["study_id"]},
                               geometry=picks, crs=pop_points.crs)
        vals = exposure_at(gdf)
        draws.append(pd.DataFrame({"study_id": gdf["study_id"], "draw": d,
                                   "exposure": vals}))
    out = pd.concat(draws, ignore_index=True)
    summary = out.groupby("study_id")["exposure"].agg(["mean", "std", "count"])
    log.info("imputed %d records over %d draws; median within-record sd=%.4f",
             len(summary), n_draws, float(summary["std"].median()))
    return out

The missingness profile is the step that decides everything downstream and is skipped most often. It compares assignable and unassignable records on every covariate the file contains — age, sex, rurality, case status, reporting source. A profile where the two groups look alike on everything supports complete-case analysis. A profile where unassignable records are twice as likely to be rural, which is the usual result, does not.

The imputation draws a different plausible location on each draw and computes the exposure there, so the between-draw variation in the pooled estimate reflects the genuine uncertainty about where the person lived. Assigning the area mean once, by contrast, pretends that location is known to be the average. The distinction matters most where exposure varies sharply within the area, which is exactly where the assignment was needed.

Comparing the Three Strategies on One Analysis Three effect estimates with confidence intervals from the same study, differing only in how unassignable records were handled. The complete-case estimate is the largest with the narrowest interval, but rests on the smallest and least representative sample. The coarse assignment estimate is smaller because everyone in an area shares one exposure value, diluting the contrast, and its interval is similar in width. The multiple imputation estimate lies between the other two with a visibly wider interval that reflects the uncertainty about where the unassignable records actually lived. A note states that reporting all three is the point. Same data, three handling rules, three different published numbers no effect complete case n = 41 200 coarse assignment n = 48 900 multiple imputation n = 48 900 The wider interval is not a worse result — it is the honest one the narrow complete-case interval hides the uncertainty rather than removing it

Validation & Edge Cases

A containing geography that is too large to impute within. If the only known geography is a county, sampling within it produces draws that are nearly unrelated to the truth, and the imputation degenerates to a noisy version of coarse assignment. Set a maximum area above which imputation is not attempted.

Population weights that are the wrong vintage. Imputing 2010 residential locations for a 2023 record puts people in subdivisions that did not exist, or misses ones that do. Match the population surface vintage to the record.

Records with no geography at all. Some have neither coordinate nor area. These cannot be imputed by any method and must be excluded, counted and reported separately from the imputable ones.

Imputation used to rescue a very high missing fraction. Above roughly a quarter of records, the imputed values dominate the exposure distribution and the analysis is largely modelling assumptions. Improve the geocoding first — address normalisation and a second reference source usually recover a large share.

Imputation that ignores what else is known about the record. A record with no coordinate often still carries a reported city, a school district, or a facility of diagnosis. Sampling uniformly across the whole county when the city is known throws away real information and widens the interval past what the data justify. Build the sampling pool from the narrowest geography actually present per record, and record which geography each imputation used, because the mix of narrow and wide pools determines how much of the final interval is genuine uncertainty.

Treating the exclusion as reversible later. Excluded records are usually dropped early, in an extract step, and by the time the analysis is written the counts have been lost. Keep the unassignable records in the analytic file with a null exposure and an explicit reason code rather than filtering them out, so the profile can be regenerated at any point and the sensitivity analysis does not require re-running the geocoder.

Failing to pool correctly. The point of multiple imputation is that the between-draw variance enters the standard error. Averaging the draws into a single exposure column and fitting once throws that away and reports a complete-case interval on imputed data.

INFO exposure.missing missingness: 6142/48900 unassignable (12.6%); largest covariate gap: rural
INFO exposure.missing imputed 6142 records over 20 draws; median within-record sd=1.8830

The narrow-pool refinement is worth showing, because it is the difference between an imputation that uses what is known and one that discards it:

Sample From the Narrowest Geography You Actually Know Three nested sampling pools for an unassignable record: the whole county, the reported city within it, and the school district within the city. Beside each pool is the spread of imputed exposure values drawn from it. The county pool produces a wide spread because it mixes very different environments; the city pool is narrower; the school district pool is narrowest. Using the county when the city is known inflates the interval without any justification in the data. A wider pool is not more cautious, it is less informed county reported city school district county pool: widest spread city pool district pool: narrowest spread of imputed exposure Record which geography each record's imputation used

Compliance Notes

Record the geocode success rate by match type, the definition of assignable used for this analysis, the missingness profile table comparing assignable and unassignable records on every available covariate, the strategy chosen with its justification, the imputation seed, draw count and population surface vintage, and the results of every strategy that was run.

Report the missingness profile in any published methods section. A study that states only its final sample size has not given a reader the means to judge who is missing from it, and in exposure work the missing are reliably the rural, the recently arrived and the poorly housed — precisely the groups an environmental justice analysis exists to describe.