Spatial Epi Public Health GIS

Estimating Intercensal Population for Trend Analysis

This guide, part of Small-Area Population Denominators, produces a year-by-year small-area population series between two censuses — the denominator a ten-year rate trend needs and neither census can supply on its own.

Problem Context & Constraints

A rate trend from 2010 to 2020 at tract level has a denominator problem at both ends. The 2010 census gives an exact count for 2010 that grows stale through the decade. The 2020 census gives an exact count for 2020 that did not exist before. A series that uses the 2010 count until 2019 and switches to the 2020 count in 2020 puts a step into every rate at the switch: a tract that grew by 20% over the decade shows a rate 20% too high by 2019, then drops abruptly when the denominator catches up. The step looks like a decline in disease, and it is entirely an artefact.

Intercensal estimation replaces the two point measurements with a smooth path between them, anchored exactly at both census dates. It is a model, not a measurement, and it has to be treated as one: its assumption is that population changed smoothly, which is wrong wherever a large development opened or a facility closed mid-decade.

Three constraints shape a defensible method. The estimates must be anchored exactly at both census counts, so the series agrees with the enumerations. Small-area estimates should sum to published county estimates where those exist for each year, because county estimates use administrative data — births, deaths, migration — that small-area interpolation cannot see. And boundary changes between censuses must be resolved first, since the two counts are only comparable on common geography.

The Step a Vintage Switch Puts Into a Trend A tract's rate over eleven years. Using the 2010 census denominator for every year until 2019, the rate climbs as the stale denominator falls further behind a growing population, then drops sharply in 2020 when the new census count is adopted. Using intercensal denominators that move smoothly from the 2010 count to the 2020 count, the rate series is flat, revealing that the apparent rise and sudden fall were produced by the denominator rather than by disease. The rise and the drop are both the denominator stale 2010 denominator switch intercensal denominators: flat 2010 2015 2020 rate for a tract that grew 20% over the decade, true risk constant rate

Prerequisites

  • python 3.11
  • pandas 2.2.2, numpy 1.26.4
  • Census counts at both endpoints on common, harmonised geography — see Crosswalking 2010 to 2020 Census Tracts for Trend Analysis
  • Annual published county population estimates for every intervening year
  • Optionally, an annual small-area indicator such as housing unit counts, to shape the path

Step-by-Step Solution

Interpolate between the anchors — geometrically by default, or shaped by a housing indicator where one exists — then rake each year’s small-area estimates to the published county total.

# Intercensal small-area population: anchored interpolation raked to county totals.
# Pinned: pandas==2.2.2, numpy==1.26.4
import logging

import numpy as np
import pandas as pd

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("intercensal")


def interpolate(anchors: pd.DataFrame, years: range, shape: pd.DataFrame | None = None,
                method: str = "geometric") -> pd.DataFrame:
    """anchors: geoid, county, pop_start, pop_end (common geography)
    shape:   optional geoid, year, indicator (e.g. housing units) to shape the path
    """
    y0, y1 = years.start, years.stop - 1
    rows = []
    for r in anchors.itertuples():
        for y in years:
            t = (y - y0) / (y1 - y0)
            if shape is not None:
                s = shape[(shape["geoid"] == r.geoid)].set_index("year")["indicator"]
                if y0 in s and y1 in s and s[y1] != s[y0]:
                    t = float(np.clip((s.get(y, np.nan) - s[y0]) / (s[y1] - s[y0]), 0, 1))
            if method == "geometric" and r.pop_start > 0 and r.pop_end > 0:
                p = r.pop_start * (r.pop_end / r.pop_start) ** t
            else:
                p = r.pop_start + t * (r.pop_end - r.pop_start)
            rows.append({"geoid": r.geoid, "county": r.county, "year": y, "pop": p})
    df = pd.DataFrame(rows)
    log.info("interpolated %d areas over %d years (%s%s)", anchors.shape[0], len(years),
             method, ", shaped by indicator" if shape is not None else "")
    return df


def rake_to_county(est: pd.DataFrame, county_totals: pd.DataFrame) -> pd.DataFrame:
    """Scale each county-year's areas so they sum to the published county estimate.

    Census years are left untouched so the anchors stay exact.
    """
    s = est.groupby(["county", "year"])["pop"].sum().rename("sum_est").reset_index()
    j = est.merge(s, on=["county", "year"]).merge(county_totals, on=["county", "year"],
                                                  how="left")
    anchor_years = {est["year"].min(), est["year"].max()}
    factor = np.where(j["year"].isin(anchor_years) | j["county_total"].isna(), 1.0,
                      j["county_total"] / j["sum_est"])
    j["pop_raked"] = j["pop"] * factor
    adj = pd.Series(factor[~j["year"].isin(anchor_years)])
    log.info("raked to county totals: median factor=%.4f, range %.4f..%.4f",
             adj.median(), adj.min(), adj.max())
    if (adj - 1).abs().max() > 0.05:
        log.warning("some county-years needed >5%% adjustment — check the anchors and "
                    "the boundary harmonisation for those counties")
    return j[["geoid", "county", "year", "pop", "pop_raked"]]

Raking to county totals is what brings administrative information into a small-area series that otherwise knows only two numbers. A county’s annual estimate reflects births, deaths and migration each year; raking forces the tracts inside it to agree with that total while preserving their relative shares from the interpolation. The anchor years are exempted so the census counts remain exact.

The size of the raking factor is a diagnostic. Factors close to one mean the interpolation and the county estimate agree. A factor that departs substantially from one says the smooth path is missing something real — a recession, a large development, an institutional closure — and the tracts in that county deserve a closer look before their rates are trusted.

Three Paths Between the Same Two Anchors A tract's population between two census counts, drawn under three interpolation methods. Linear interpolation rises in a straight line. Geometric interpolation rises along a gentle curve implied by a constant growth rate. Interpolation shaped by housing unit counts stays nearly flat for five years and then rises steeply after a subdivision was completed, which is what actually happened. All three meet both anchors exactly and differ in the intervening years by up to a fifth of the population, which is the size of the rate error the choice can introduce. Same anchors, different paths, different mid-decade rates linear geometric housing-shaped subdivision completed 2010 census 2020 census tract population between two anchors

Validation & Edge Cases

Boundary changes not resolved. Interpolating between a 2010 tract count and a 2020 count for a tract that was split in 2020 produces nonsense. Harmonise both censuses to one geography first; the crosswalk’s own allocation error then enters the series.

Abrupt changes smoothed away. A facility that opened in 2016 appears as a steady rise from 2010. Where an indicator such as housing units or group-quarters counts exists, use it to shape the path; where not, flag tracts with large decade change for manual review.

Negative or zero populations. Linear interpolation between a positive and zero endpoint — a demolished housing project — produces tiny populations in late years with enormous rates. Treat those tracts separately rather than letting the arithmetic run.

County estimates revised after the fact. Published county estimates are revised with each vintage. Rake to one declared vintage of county estimates and record it; re-raking to a later vintage changes every small-area value.

Post-censal years. Extending the method beyond the last census requires projection, not interpolation, and the uncertainty grows each year. Label post-censal estimates as such and carry wider uncertainty.

INFO intercensal interpolated 1842 areas over 11 years (geometric, shaped by indicator)
INFO intercensal raked to county totals: median factor=0.9981, range 0.9412..1.0634
WARNING intercensal some county-years needed >5% adjustment — check the anchors and the boundary harmonisation for those counties

Mixing intercensal and post-censal estimates in one series. A trend from 2015 to 2025 spans the 2020 census, so its early years can be intercensal while its later years can only be post-censal projections. The two have different error properties and will be revised differently when the next census arrives. Carry a column indicating which kind each year is, and treat the post-censal end of any trend line as provisional.

Uncertainty that grows between the anchors. The estimate is exact at both census years and least certain in the middle of the decade, where it is furthest from either measurement. A trend analysis that weights every year equally treats the mid-decade denominators as though they were as good as the census ones. Where the analysis supports it, carry an uncertainty that peaks mid-decade and falls to zero at the anchors, and prefer methods that down-weight the least certain years.

Tracts that swap between urban and rural character. A tract that urbanises over the decade changes not only its population but its age structure, and a total-population interpolation says nothing about the latter. Where rates are age-standardised, interpolate each age band separately or accept that the stratified denominators are less reliable than the total in exactly the tracts that changed most.

The uncertainty shape described above is worth drawing, because it is the reason the anchor years deserve more weight than the rest:

Exact at the Ends, Least Certain in the Middle An intercensal population estimate drawn as a line between two census anchors, with an uncertainty band around it. The band has zero width at both census years, where the estimate equals the enumeration, and widens to its maximum around the middle of the decade, furthest from either measurement. Raking to county totals narrows the band somewhat but does not remove its shape. A trend analysis should weight mid-decade years less than the anchor years, or at least carry this band into its intervals. Mid-decade denominators are the least certain in the series widest mid-decade census census intercensal estimate with its uncertainty band

Compliance Notes

Record the anchor censuses and their geography, the crosswalk used to harmonise them, the interpolation method and any shaping indicator with its source, the county estimate vintage used for raking, the raking factors by county and year, and the tracts flagged for manual review.

Trend products built on intercensal denominators should say so, and should identify the post-censal years whose denominators are projections. When the next census arrives, the intercensal series for the preceding decade will be re-estimated and every rate in it will move slightly — a known, scheduled revision that should be anticipated in the product’s documentation rather than surprising the partners who cited the earlier values.