Spatial Epi Public Health GIS

Building Synthetic Case Data for Pipeline Tests

This guide, part of Testing Geospatial Analysis Code, builds the fixture a surveillance pipeline needs to be testable by anyone: a complete, realistic-shaped dataset with known answers, generated from a seed, containing no real person.

Problem Context & Constraints

The pipeline code that most needs testing — geocoding, exposure joins, suppression — handles addresses and case locations. That makes it the code most likely to be tested only inside the protected environment, by whoever has access, on real data, with no known right answer. The consequence is predictable: those stages accumulate bugs that nobody outside a small group can find.

The obvious shortcut — jittering real case locations to make a “de-identified” fixture — is a disclosure. A jittered point set is a geomasked release, and it will be committed to a repository with far broader access than the case file, copied into continuous integration logs, and forked by contractors. It is exactly the product the privacy section of this site describes how to review, created without review.

A synthetic fixture avoids both problems, provided it meets three constraints. It must be realistic in the ways the code cares about — irregular polygons with islands and slivers, a population surface with urban and rural contrast, addresses in messy formats — or the tests will pass on easy inputs. It must carry known answers — a planted cluster at a known place, a known true rate per area, a known number of unassignable records — so tests can assert correctness rather than only stability. And it must be generated from a seed, so it is reproducible and so any fixture can be regenerated rather than stored.

One Seed, Five Layers, Every Answer Known A stack of generated layers from a single seed. First a synthetic geography of irregular areas nested into larger districts, including an island and a sliver. Second a population surface with an urban core and rural periphery. Third background cases drawn in proportion to population, giving a known true rate per area. Fourth a planted cluster at a recorded centre and radius. Fifth an address layer in which a known share are deliberately malformed or unassignable. Beside each layer is the known answer it lets a test assert. Every layer is generated, so every correct answer is known 1 · Geography with islands and slivers known: nesting, neighbours, islands 2 · Population surface known: denominator per area 3 · Background cases at a set rate known: true rate everywhere 4 · Planted cluster known: centre, radius, excess 5 · Messy addresses known: share unassignable, by type seed 20260918 regenerates all five

Prerequisites

  • python 3.11
  • numpy 1.26.4, geopandas 0.14.4, shapely 2.0.4, pandas 2.2.2, scipy 1.13.1
  • The pipeline’s working coordinate reference system
  • A list of the address malformations the real feed produces, from the geocoding error log
  • The detection effect sizes the pipeline is expected to find, so the planted cluster is realistic

Step-by-Step Solution

Generate the layers in order from one seed, recording every known answer in a manifest that tests read.

# Seeded synthetic surveillance fixture with a manifest of known answers.
# Pinned: numpy==1.26.4, geopandas==0.14.4, shapely==2.0.4, pandas==2.2.2, scipy==1.13.1
import json
import logging

import numpy as np
import pandas as pd
import geopandas as gpd
from scipy.spatial import Voronoi
from shapely.geometry import Point, Polygon, box

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("fixtures.synthetic")

CRS = "EPSG:5070"
EXTENT = (0.0, 0.0, 60_000.0, 60_000.0)


def synthetic_areas(rng, n_areas=60, n_districts=6) -> gpd.GeoDataFrame:
    """Irregular areas from a clipped Voronoi tessellation, nested into districts."""
    minx, miny, maxx, maxy = EXTENT
    seeds = rng.uniform([minx, miny], [maxx, maxy], size=(n_areas, 2))
    pad = np.array([[minx - 1e5, miny - 1e5], [maxx + 1e5, miny - 1e5],
                    [minx - 1e5, maxy + 1e5], [maxx + 1e5, maxy + 1e5]])
    vor = Voronoi(np.vstack([seeds, pad]))
    frame = box(*EXTENT)
    polys = []
    for i in range(n_areas):
        region = vor.regions[vor.point_region[i]]
        if -1 in region or not region:
            continue
        polys.append(Polygon(vor.vertices[region]).intersection(frame))
    gdf = gpd.GeoDataFrame({"geoid": [f"A{i:03d}" for i in range(len(polys))]},
                           geometry=polys, crs=CRS)
    # Districts: group areas by k-means-like nearest seed, giving clean nesting.
    dseeds = rng.uniform([minx, miny], [maxx, maxy], size=(n_districts, 2))
    c = np.c_[gdf.geometry.centroid.x, gdf.geometry.centroid.y]
    gdf["district"] = [f"D{np.argmin(((dseeds - p) ** 2).sum(1))}" for p in c]
    # One offshore island and one sliver, deliberately.
    island = box(maxx + 2_000, maxy - 8_000, maxx + 6_000, maxy - 4_000)
    sliver = box(minx + 30_000, miny, minx + 30_004, miny + 5_000)
    extra = gpd.GeoDataFrame({"geoid": ["ISLAND", "SLIVER"],
                              "district": ["D0", "D1"]},
                             geometry=[island, sliver], crs=CRS)
    return pd.concat([gdf, extra], ignore_index=True)


def population(rng, areas: gpd.GeoDataFrame, core=(20_000.0, 35_000.0)) -> pd.Series:
    """Denser near an urban core; the known denominator per area."""
    c = areas.geometry.centroid
    d = np.hypot(c.x - core[0], c.y - core[1])
    base = 25_000 * np.exp(-d / 12_000) + 400
    return pd.Series(np.round(base * rng.uniform(0.8, 1.2, len(areas))).astype(int),
                     index=areas.index, name="population")


def cases(rng, areas, pop, rate=0.002, cluster=None) -> tuple[gpd.GeoDataFrame, dict]:
    """Background cases at a fixed rate plus an optional planted cluster."""
    pts, planted = [], []
    for idx, row in areas.iterrows():
        n = rng.poisson(rate * pop[idx])
        minx, miny, maxx, maxy = row.geometry.bounds
        while n > 0:
            p = Point(rng.uniform(minx, maxx), rng.uniform(miny, maxy))
            if row.geometry.contains(p):
                pts.append(p); planted.append(False); n -= 1
    truth = {"background_rate": rate}
    if cluster:
        cx, cy, r, k = cluster["x"], cluster["y"], cluster["radius"], cluster["n"]
        rr = r * np.sqrt(rng.uniform(0, 1, k)); tt = rng.uniform(0, 2 * np.pi, k)
        for a, b in zip(cx + rr * np.cos(tt), cy + rr * np.sin(tt)):
            pts.append(Point(a, b)); planted.append(True)
        truth["cluster"] = cluster
    gdf = gpd.GeoDataFrame({"case_id": [f"C{i:06d}" for i in range(len(pts))],
                            "planted": planted}, geometry=pts, crs=CRS)
    return gdf, truth


MALFORMATIONS = ("po_box", "rural_route", "missing_number", "zip_only", "clean")


def addresses(rng, n: int, shares=(0.03, 0.02, 0.05, 0.04, 0.86)) -> pd.DataFrame:
    """Synthetic address strings with a KNOWN share of each malformation."""
    kinds = rng.choice(MALFORMATIONS, size=n, p=shares)
    num = rng.integers(1, 9999, n)
    street = rng.choice(["OAK", "MAPLE", "RIVER", "HILL", "MAIN"], n)
    text = []
    for k, a, s in zip(kinds, num, street):
        text.append({"po_box": f"PO BOX {a}", "rural_route": f"RR {a % 9 + 1} BOX {a}",
                     "missing_number": f"{s} ST", "zip_only": "",
                     "clean": f"{a} {s} ST"}[k])
    return pd.DataFrame({"address": text, "kind": kinds})


def build_fixture(seed: int, out_dir: str) -> dict:
    rng = np.random.default_rng(seed)
    areas = synthetic_areas(rng)
    areas["population"] = population(rng, areas)
    cluster = {"x": 41_000.0, "y": 18_000.0, "radius": 1_500.0, "n": 90}
    pts, truth = cases(rng, areas, areas["population"], cluster=cluster)
    addr = addresses(rng, len(pts))
    pts = pts.join(addr)
    manifest = {"seed": seed, "crs": CRS, "n_areas": int(len(areas)),
                "n_cases": int(len(pts)), "truth": truth,
                "unassignable_kinds": ["po_box", "rural_route", "zip_only"],
                "expected_unassignable": int(addr["kind"].isin(
                    ["po_box", "rural_route", "zip_only"]).sum()),
                "islands": ["ISLAND"], "slivers": ["SLIVER"]}
    areas.to_parquet(f"{out_dir}/areas.parquet")
    pts.to_parquet(f"{out_dir}/cases.parquet")
    with open(f"{out_dir}/manifest.json", "w") as fh:
        json.dump(manifest, fh, indent=2)
    log.info("fixture seed=%d areas=%d cases=%d unassignable=%d",
             seed, manifest["n_areas"], manifest["n_cases"],
             manifest["expected_unassignable"])
    return manifest

The manifest is what makes the fixture a test oracle rather than just test data. Every property a test might want to assert — the planted cluster’s location, the expected number of unassignable records, which areas are islands — is written down at generation time, so a test compares the pipeline’s output to a known answer instead of to “whatever it produced last time”.

The deliberate island and sliver matter as much as the planted cluster. Real boundary files contain both, and the spatial weights and geometry checks that fail on them are exactly the code a clean, all-convex synthetic geography never exercises.

Why Synthetic Beats Jittered Real Data Three comparisons between a fixture made by jittering real case points and a fully synthetic fixture. On disclosure, the jittered fixture is a masked release requiring review and the synthetic one contains nobody. On known answers, the jittered fixture has none because the truth is the real data, while the synthetic one records every answer in its manifest. On reach, the jittered fixture can only be used by people with protected access, while the synthetic one can be committed, run in continuous integration and shared with contractors. A jittered real dataset is a release, not a fixture jittered real points fully synthetic disclosure risk a masked release contains nobody known answers none recorded all in the manifest who can run tests protected users only anyone, in CI regenerable no, it must be stored yes, from the seed Every row favours synthetic; realism is the only thing it has to earn

Validation & Edge Cases

Fixtures that are too clean. A synthetic geography of Voronoi cells with no islands, slivers or multipart features lets the weights and overlay code pass untested. Add the awkward cases deliberately, and add a new one each time a real defect is found.

Addresses that do not resemble the feed. The malformation list should come from the geocoding stage’s actual error log, not from imagination. When the real feed develops a new failure pattern, add it to the generator.

Planted effects that are trivially strong. A cluster of 500 extra cases on a background of 100 tests nothing. Size the planted excess at the effect the pipeline is expected to detect, and add one below the limit to confirm the null behaviour.

Real place names and street names. Synthetic addresses using a real street name and a real house-number range in a real town are closer to a real address than they look. Use invented names, or a coordinate system that does not correspond to any real place.

Fixture drift. If the generator changes, the fixture changes, and stored regression outputs become invalid. Version the generator and record its version in the manifest.

INFO fixtures.synthetic fixture seed=20260918 areas=62 cases=1184 unassignable=108

Sizing the planted cluster is a calibration exercise in its own right, and the useful sizes bracket the method’s detection limit:

Plant Effects on Both Sides of the Detection Limit An S-shaped curve of detection probability against the size of the planted excess. Very small excesses are almost never detected and very large ones almost always are. The useful fixtures sit on either side of the steep middle: one planted comfortably above the limit, which the method must find, and one just below it, which the method should usually miss. A trivially large planted cluster in the flat upper region tests nothing, because any method would pass. A trivially strong planted cluster tests nothing just below: should miss above: must find trivial: tests nothing size of the planted excess detection probability

Compliance Notes

Record the generator version, the seed, the manifest and the source of the malformation list for every fixture. The fixture itself contains no protected information and can be committed, shared with contractors and run in any continuous integration service.

State in the repository that fixtures are fully synthetic and must remain so. The erosion happens gradually — someone adds “a few real problem addresses to make the test realistic” — and it converts a safe repository into a disclosure. A review rule that any fixture change must regenerate from the seeded generator, rather than hand-editing the data files, prevents it structurally.