Spatial Epi Public Health GIS

Assigning Exposure With Residential Mobility Histories

This guide, part of Assigning Exposure to Residential Locations, handles the case where a single address cannot represent the exposure window: a cancer with a twenty-year latency, a birth outcome whose relevant window is one trimester, a chronic condition whose registry record holds only the address at diagnosis.

Problem Context & Constraints

A surveillance record usually carries one address. For an acute outcome, that address is close enough to the exposure period to be defensible. For anything with a latency measured in years, it is the address where the person was living when they were diagnosed, which may be a retirement move hundreds of kilometres from where the exposure occurred. Roughly a third to a half of a population changes address within five years, so this is the common case rather than the exception.

The naive treatment — use the address on file and note the limitation — introduces error whose structure depends on why people moved. If moves are unrelated to exposure, the result is a diluted contrast and attenuation toward the null. If moves are related to exposure, as they are when people leave a neighbourhood because of a facility, the error is differential and the bias can go either way.

Building a time-weighted exposure from a residential history fixes the structure of the problem and introduces two of its own. Histories are incomplete, with gaps of unknown address covering months or years. History sources are biased: commercial address histories cover homeowners and people with credit records better than renters and young adults, so the people with the best histories are not a random sample.

The constraint that keeps the method honest is that a time-weighted exposure must carry a coverage measure — what fraction of the relevant window has a known address — and that coverage must enter the analysis rather than being averaged away.

A Residential History Across an Exposure Window A timeline running from twenty years before diagnosis to diagnosis. Three residential periods are drawn as bars with different exposure levels, separated by two gaps where no address is known. The time-weighted mean is computed over the covered periods only and shown as a horizontal line. The value at the address on file at diagnosis is shown as a separate, much lower line, because the final residence has low exposure and covers only the last three years of the window. The gap periods are annotated with the coverage fraction, and a note states that a history covering half the window supports a weaker claim than one covering all of it. The address at diagnosis describes the last few years, not the window 20 years before 10 years before diagnosis residence A — high gap residence B — medium gap residence C — low time-weighted mean over covered periods value at the address on file coverage here is 80% of the window — record it, and let the analysis weight by it

Prerequisites

  • python 3.11
  • pandas 2.2.2, numpy 1.26.4, geopandas 0.14.4
  • A residential history table with one row per address period: study identifier, start date, end date, geocode and match type
  • Exposure surfaces for each year or period of the window, not a single surface
  • An explicit definition of the exposure window relative to the outcome date

Step-by-Step Solution

Intersect each residential period with the exposure window, assign the surface of the right vintage to each intersection, and take a duration-weighted mean over what is covered.

# Time-weighted residential exposure from an address history.
# Pinned: pandas==2.2.2, numpy==1.26.4, geopandas==0.14.4
import logging

import numpy as np
import pandas as pd

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("exposure.mobility")

MIN_COVERAGE = 0.5      # below this the time-weighted value is not reported


def exposure_window(outcome_date: pd.Series, lag_years: float,
                    duration_years: float) -> pd.DataFrame:
    """Window [outcome - lag - duration, outcome - lag] for each subject."""
    end = outcome_date - pd.to_timedelta(lag_years * 365.25, unit="D")
    start = end - pd.to_timedelta(duration_years * 365.25, unit="D")
    return pd.DataFrame({"win_start": start, "win_end": end})


def time_weighted_exposure(history: pd.DataFrame, window: pd.DataFrame,
                           annual_exposure: pd.DataFrame) -> pd.DataFrame:
    """Duration-weighted mean exposure over the covered part of the window.

    history: study_id, addr_start, addr_end, addr_id, geocode_match
    annual_exposure: addr_id, year, exposure  (surface of that year's vintage)
    """
    h = history.merge(window, left_on="study_id", right_index=True, how="inner")
    # Clip each residential period to the exposure window.
    h["seg_start"] = h[["addr_start", "win_start"]].max(axis=1)
    h["seg_end"] = h[["addr_end", "win_end"]].min(axis=1)
    h = h[h["seg_end"] > h["seg_start"]].copy()

    # Split each clipped period at year boundaries so the right surface applies.
    rows = []
    for rec in h.itertuples():
        y0, y1 = rec.seg_start.year, rec.seg_end.year
        for yr in range(y0, y1 + 1):
            s = max(rec.seg_start, pd.Timestamp(year=yr, month=1, day=1))
            e = min(rec.seg_end, pd.Timestamp(year=yr + 1, month=1, day=1))
            days = (e - s).days
            if days <= 0:
                continue
            rows.append({"study_id": rec.study_id, "addr_id": rec.addr_id,
                         "year": yr, "days": days,
                         "geocode_match": rec.geocode_match})
    seg = pd.DataFrame(rows)
    if seg.empty:
        log.warning("no residential period overlaps the exposure window")
        return pd.DataFrame(columns=["study_id", "exposure_twa", "coverage"])

    seg = seg.merge(annual_exposure, on=["addr_id", "year"], how="left")
    covered = seg.dropna(subset=["exposure"])

    win_days = (window["win_end"] - window["win_start"]).dt.days
    agg = covered.groupby("study_id").apply(
        lambda d: pd.Series({
            "exposure_twa": float(np.average(d["exposure"], weights=d["days"])),
            "covered_days": int(d["days"].sum()),
            "n_addresses": int(d["addr_id"].nunique()),
        }), include_groups=False).reset_index()
    agg = agg.merge(win_days.rename("window_days"), left_on="study_id",
                    right_index=True, how="left")
    agg["coverage"] = agg["covered_days"] / agg["window_days"]
    agg.loc[agg["coverage"] < MIN_COVERAGE, "exposure_twa"] = np.nan

    log.info("time-weighted exposure: n=%d median_coverage=%.2f "
             "below_min_coverage=%d median_addresses=%.1f",
             len(agg), agg["coverage"].median(),
             int((agg["coverage"] < MIN_COVERAGE).sum()),
             agg["n_addresses"].median())
    return agg[["study_id", "exposure_twa", "coverage", "n_addresses"]]

Splitting each residential period at year boundaries is what makes the surface vintage correct. Using a single surface for a twenty-year window assigns 2005 concentrations to a 2005 address and also to a 2020 one, which discards the secular trend that is often the largest source of variation in the whole calculation — roadside nitrogen dioxide in many cities has fallen by a third over such a period.

The coverage threshold nulls the value rather than reporting a time-weighted mean based on a fifth of the window. A mean over 18% of the exposure period is not a summary of that period; it is a summary of whichever part of it the address history happened to capture, and those two are only the same thing if the missing part was missing at random.

What Mobility Costs a Single-Address Analysis Two curves showing the magnitude of an estimated exposure effect against the share of the cohort that moved during the exposure window. Using only the address at diagnosis, the estimate shrinks steadily toward the null as the mover share grows, because more of the cohort is assigned an exposure unrelated to their window. Using a time-weighted history, the estimate stays close to its true value until the mover share is very high, where incomplete coverage begins to matter. A note records that the two curves coincide when nobody moves, which is why single-address analyses look adequate in short-latency studies. Mobility attenuates a single-address estimate in proportion to itself true effect time-weighted history address at diagnosis only 0% 40% 80% share of the cohort that moved during the exposure window Report the mover share: it bounds how much a single-address result could be understating estimated effect

Validation & Edge Cases

Overlapping or contradictory address periods. Commercial histories routinely report two addresses simultaneously. Resolve with an explicit rule — longest period wins, or most recent source wins — and record how many records needed it.

Gaps that are not missing at random. People are hardest to track during periods of housing instability, which correlate with both exposure and outcome. Report coverage by case status; a systematic difference is a limitation that must be stated.

Histories that begin at the wrong end. Many sources start at the first credit record, so childhood addresses are absent. For an outcome with a childhood-exposure hypothesis the history is structurally uninformative, whatever its coverage number says.

Moves within the same exposure level. Not every move changes exposure. A household that relocates three streets away in the same neighbourhood contributes an address change and no exposure change, while a cross-state move changes everything. Summarising mobility as a count of addresses therefore overstates its importance. Report instead the within-subject standard deviation of assigned exposure across the window — that is the quantity that determines how much a single-address analysis could have lost, and it is frequently near zero for a large share of a cohort.

Reconstructing periods from an employer or insurer file. Administrative sources sometimes carry an address effective date but no end date, and the naive reconstruction assumes each address runs until the next one appears. That is usually right and occasionally very wrong, when a record was simply not updated for years. Cap the inferred duration at a stated maximum, count how many periods were capped, and treat a record dominated by capped periods as low coverage regardless of what the raw arithmetic says.

Deciding the lag and duration after seeing results. The exposure window has two free parameters, and sweeping them produces as many estimates as a buffer ladder does. Declare the primary window from the disease’s biology, report alternatives as a pre-listed sensitivity set, and keep the primary result the headline.

Geocode quality varying across the history. Older addresses geocode worse. The per-period assignment must carry its own match type and positional error rather than inheriting the most recent one.

No annual surfaces available. Where only one surface vintage exists, say so and report the analysis as assuming a stationary exposure field, with the secular trend quantified separately if any monitor record allows it.

INFO exposure.mobility time-weighted exposure: n=18422 median_coverage=0.78 below_min_coverage=3104 median_addresses=2.0

The coverage distribution is what decides how much of a cohort the time-weighted exposure actually describes, and it should be inspected before any model is fitted:

How Much of the Window the Histories Cover A histogram of the fraction of each subject's exposure window covered by a known address. Most subjects have high coverage, clustered above eighty per cent. A long tail of subjects has low coverage, concentrated among younger subjects and renters whose address histories are thinner. A vertical line at the fifty per cent minimum marks where time-weighted values are nulled rather than reported, and the tail to its left is the group whose exclusion must be reported by case status. Check who falls below the coverage threshold before modelling 50% minimum nulled, reported by case status 0% 100% share of each subject's exposure window with a known address

Compliance Notes

A residential history is substantially more disclosive than a single address: a sequence of addresses with dates identifies a household almost uniquely and is difficult to de-identify meaningfully. It must live inside the protected environment, be accessed under the same controls as the case file, and never appear in an output, a log line or an intermediate written to shared storage.

The record for the assignment needs the history source and its coverage characteristics, the exposure window definition with lag and duration, the rule used to resolve overlapping periods, the minimum coverage threshold, the surface vintages used per year, and the distributions of coverage and address count — including by case status, since a coverage difference between cases and non-cases is a bias that must be reported rather than discovered later.