Spatial Epi Public Health GIS

Building Age-Sex Stratified Denominators for Standardization

This guide, part of Small-Area Population Denominators, builds the stratified population table that age standardisation depends on — the one input where a small misalignment silently moves every standardised rate on the map.

Problem Context & Constraints

Age standardisation exists because crude rates confound disease risk with age structure. A retirement community and a university district with identical age-specific risks have wildly different crude rates for any age-related condition, and a crude map compares their age pyramids rather than their risk. Standardisation removes that by weighting age-specific rates to a common standard population.

It only works if the age-specific rates are right, which requires the numerator and denominator to use identical strata. That is harder than it sounds. Survey tables publish ages in groups chosen for their own purposes — under 5, 5 to 9, 10 to 14, 15 to 17, 18 and 19, 20, 21, 22 to 24 — while case data carry single-year ages and the standard population uses its own bands. Three schemes, none of which line up, and every misalignment moves population from one stratum to another without moving cases.

Three constraints govern the build. Every source age group must map to exactly one analysis band, and a source group that straddles two analysis bands must be split explicitly rather than assigned wholesale. The open-ended top band — 85 and over, say — must match across numerator, denominator and standard, because it contains a disproportionate share of deaths for many outcomes. And small areas favour indirect standardisation, because direct standardisation needs a reliable rate in every stratum, which small areas rarely have.

Three Age Schemes, One Set of Analysis Bands Three age schemes shown as horizontal segmented bars. The survey table's groups have irregular widths, including narrow groups at eighteen to twenty-one. Case data carry single years. The standard population uses its own bands. All three are mapped onto one set of analysis bands shown at the bottom. A survey group that straddles two analysis bands is highlighted and must be split proportionally rather than assigned entirely to one side, since assigning it wholesale moves population between strata without moving cases. Every scheme must map onto the same analysis bands exactly survey straddles cases single years standard analysis split straddling survey groups in proportion; never assign them wholesale to one band

Prerequisites

  • python 3.11
  • pandas 2.2.2, numpy 1.26.4
  • A population table by area, sex and source age group, with margins where it is a survey product
  • Case counts by area, sex and single-year age, or in bands that nest inside the analysis bands
  • A standard population in the analysis bands, recorded with its source
  • Stratum-specific reference rates for indirect standardisation, typically state or national

Step-by-Step Solution

Map source groups to analysis bands with explicit splits for straddlers, verify population totals are conserved, then compute both indirect and direct standardised measures and choose by area size.

# Stratified denominators and direct/indirect standardisation for small areas.
# Pinned: pandas==2.2.2, numpy==1.26.4
import logging

import numpy as np
import pandas as pd

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("standardise")


def map_to_bands(pop: pd.DataFrame, band_map: list[tuple[str, str, float]]) -> pd.DataFrame:
    """band_map rows: (source_group, analysis_band, share). Shares per source must sum to 1.

    A source group lying entirely in one band has one row with share 1.0; a group
    straddling two bands has two rows whose shares split it (e.g. by single-year
    distribution, or evenly by years if nothing better exists).
    """
    m = pd.DataFrame(band_map, columns=["source_group", "band", "share"])
    check = m.groupby("source_group")["share"].sum()
    bad = check[~np.isclose(check, 1.0)]
    if len(bad):
        raise ValueError(f"shares do not sum to one for: {bad.to_dict()}")
    unmapped = set(pop["source_group"]) - set(m["source_group"])
    if unmapped:
        raise KeyError(f"source groups with no mapping: {sorted(unmapped)}")
    j = pop.merge(m, on="source_group")
    j["population"] = j["estimate"] * j["share"]
    j["moe_part"] = j["moe"] * j["share"]
    out = j.groupby(["geoid", "sex", "band"], as_index=False).agg(
        population=("population", "sum"),
        moe=("moe_part", lambda s: float(np.sqrt((s ** 2).sum()))))
    # Conservation: banding must not create or lose people.
    before = pop.groupby("geoid")["estimate"].sum()
    after = out.groupby("geoid")["population"].sum()
    gap = (before - after).abs().max()
    if gap > 1e-6:
        raise AssertionError(f"population not conserved by banding (max gap {gap})")
    log.info("banded %d areas into %d strata; totals conserved", out["geoid"].nunique(),
             out.groupby(["sex", "band"]).ngroups)
    return out


def indirect_smr(cases: pd.DataFrame, pop: pd.DataFrame, ref_rates: pd.DataFrame) -> pd.DataFrame:
    """Observed over expected, using reference stratum rates. Stable for small areas."""
    e = pop.merge(ref_rates, on=["sex", "band"])
    e["expected"] = e["population"] * e["ref_rate"]
    exp_ = e.groupby("geoid")["expected"].sum()
    obs = cases.groupby("geoid")["cases"].sum()
    out = pd.DataFrame({"observed": obs, "expected": exp_}).fillna(0.0)
    out["smr"] = out["observed"] / out["expected"].replace(0, np.nan)
    log.info("indirect SMR for %d areas; median expected=%.1f", len(out),
             out["expected"].median())
    return out.reset_index()


def direct_rate(cases: pd.DataFrame, pop: pd.DataFrame, standard: pd.DataFrame,
                min_stratum_cases: int = 5) -> pd.DataFrame:
    """Directly standardised rate; flags areas with too many sparse strata."""
    j = cases.merge(pop, on=["geoid", "sex", "band"], how="right").fillna({"cases": 0})
    j = j.merge(standard, on=["sex", "band"])
    j["rate"] = j["cases"] / j["population"].replace(0, np.nan)
    w = j["std_weight"] / j.groupby("geoid")["std_weight"].transform("sum")
    j["wrate"] = j["rate"] * w
    out = j.groupby("geoid").agg(dsr=("wrate", "sum"),
                                 sparse=("cases", lambda c: int((c < min_stratum_cases).sum())),
                                 strata=("cases", "size")).reset_index()
    out["sparse_share"] = out["sparse"] / out["strata"]
    log.info("direct rates for %d areas; median sparse-stratum share=%.2f",
             len(out), out["sparse_share"].median())
    return out

The conservation assertion in map_to_bands is the cheapest correctness check in the whole standardisation workflow. Banding should neither create nor lose people; if the total population of an area changes during banding, a share is wrong or a group was dropped, and every rate downstream is wrong by the same fraction. It takes one line and catches the error that otherwise takes days to find.

The sparse-stratum share reported by direct_rate is what decides between the two methods for a given area. Direct standardisation multiplies each stratum’s rate by a weight; when most strata have two or three cases, those rates are unstable and the weighted sum inherits their instability. Indirect standardisation borrows stable rates from a reference population and only needs the area’s total observed count, which is why it is the usual choice below county level.

Direct Standardisation Needs Every Stratum; Indirect Needs One Total The width of the confidence interval for a standardised measure plotted against area population for the two methods. The direct method's interval widens sharply as population falls, because every age-sex stratum rate becomes unstable. The indirect method's interval widens more slowly, because it depends only on the area's total observed count set against expected counts from stable reference rates. Below a crossover population, typically around county size, indirect is the defensible choice. Below county size, indirect standardisation is usually the right call direct indirect crossover tract county state area population interval width

Validation & Edge Cases

Straddling groups split evenly by default. Splitting a survey group of 15 to 19 evenly across bands of 15 to 17 and 18 to 24 assumes a uniform age distribution within the group, which is wrong near a university or a military base. Where single-year census counts exist for the area, use them to set the shares.

Open-ended top bands that differ. A survey whose top group is 85 and over, paired with a standard whose top band is 90 and over, cannot be aligned by splitting — the population above 90 is unknown. Collapse both to the coarser top band and accept the loss of detail.

Indirect ratios compared across areas. A standardised mortality ratio compares an area with the reference, not with other areas; two ratios from populations with very different age structures are not strictly comparable with each other. That is a limitation of the method, and ranking areas by it should be done with that stated.

Reference rates from a different period. Expected counts built on reference rates from a different year inherit any secular trend between the two years, which shifts every ratio together. Use reference rates from the same period as the observed counts.

Cases in strata with zero population. A case recorded in a stratum whose denominator is zero — an 80-year-old in a tract the survey estimates has nobody over 75 — is a signal of either a denominator estimate error or a geocoding problem. Count these and investigate rather than dividing by zero.

INFO standardise banded 1842 areas into 36 strata; totals conserved
INFO standardise direct rates for 1842 areas; median sparse-stratum share=0.72
INFO standardise indirect SMR for 1842 areas; median expected=6.4
What Standardisation Removes Two areas with identical age-specific disease risk but very different age structures: a retirement community weighted toward older ages and a university district weighted toward young adults. The crude rate is several times higher in the retirement community because it has more people in the high-risk age bands. After standardisation both have a ratio of one, correctly showing no difference in underlying risk. A crude-rate map would have shown the retirement community as a hotspot of disease rather than of age. A crude map compares age pyramids, not risk Retirement community crude rate: 42 per 1 000 standardised ratio: 1.00 many residents in high-risk ages University district crude rate: 9 per 1 000 standardised ratio: 1.00 many residents in low-risk ages identical age-specific risk, very different crude rates, equal standardised ratios

Compliance Notes

Record the band mapping with every share and its basis, the conservation check result, the open-ended top band used, the standard population and its source, the reference rates and their period for indirect measures, the method chosen per area with the sparse-stratum share that drove the choice, and any cases found in zero-population strata.

Standardised rates are compared across agencies more than almost any other surveillance product, and two agencies standardising the same data to different standards or bands will publish different numbers for the same county. Stating the standard population and the bands in the product metadata is what makes those comparisons resolvable rather than a source of recurring confusion.