Spatial Epi Public Health GIS

Tuning the Temporal Window for Space-Time Scans

The maximum temporal window decides what a space-time scan is able to see, and it is usually left at a library default. A window too short cannot accumulate enough excess to reach significance; a window too long dilutes a sharp signal across weeks of ordinary data. This guide, part of Space-Time Cluster Detection, sets out how to choose both bounds from the epidemiology and how to verify the choice by replaying real outbreaks.

Problem Context & Constraints

The scan evaluates every window from the minimum to the maximum length, so both bounds shape the search. The maximum matters most, and its effect is not monotone.

Consider a point-source outbreak producing forty excess cases over six days. A scan with a maximum window of seven days can place a cylinder tightly around it, and the likelihood ratio is large because the expected count in six days is small. A scan with a maximum window of ninety days will also consider a six-day window — it is inside the range — but its null distribution is now built from a much larger family of cylinders, so the same observed excess has to beat a stronger maximum under permutation. Widening the maximum therefore reduces sensitivity to short sharp events even though it does not remove them from the search.

The opposite failure is equally real. A slow diffuse increase over ten weeks is invisible to a scan whose maximum window is fourteen days, because no evaluated cylinder contains enough of it.

The minimum window has a smaller but specific role: it suppresses one-day artefacts, and where reporting is batched weekly it should be at least seven days, because a one-day window on batched data measures the batch schedule.

The Two Outbreak Shapes Want Opposite Windows Detection power plotted against the scan's maximum temporal window for two outbreak shapes. For a short sharp point-source outbreak, power peaks near a maximum window of ten days and declines steadily as the window widens, because the null distribution grows. For a slow diffuse increase, power is near zero at short maximum windows, rises steeply between twenty and fifty days, and plateaus. No single maximum serves both, which is why some systems run two configurations in parallel. One parameter, two incompatible optima 0 0.5 1.0 detection power short sharp outbreak — best near 10 days slow diffuse rise — needs 50+ 3 20 60 90 maximum temporal window (days) Running two configurations in parallel is cheaper than compromising on one

Prerequisites

  • A working prospective or retrospective scan, per the parent guide
  • At least two historical outbreaks with known onset dates and affected areas, for replay
  • A quiet period of at least a year for false-alarm measurement
  • python 3.11, numpy 1.26.4, pandas 2.2.2

The two bounds interact with the reporting cadence, and mismatching them produces an artefact that is easy to mistake for a signal:

Minimum Window Against Reporting Cadence Three combinations of minimum window and reporting cadence. A one-day minimum on daily reporting is fine. A one-day minimum on weekly batched reporting produces cylinders containing a single batch, so seventy-eight percent of alarms have a one-day window and the scan is detecting the batch schedule. A seven-day minimum on weekly batched reporting removes the artefact entirely. Match the minimum window to how the data arrives min 1 day, daily data fine min 1 day, weekly batches 78% of alarms are 1-day windows min 7 days, weekly batches artefact removed The middle panel is a scan detecting the laboratory’s courier schedule

Step-by-Step Solution

Choose the bounds from the epidemiology first, then verify by replay.

# Sweep maximum window length and score detection delay against alarm rate.
# Pinned: numpy==1.26.4, pandas==2.2.2
import logging
import numpy as np
import pandas as pd

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("scan.window")


def replay_detection_delay(cases: pd.DataFrame, outbreak, max_days: int,
                           threshold: float, runner) -> int | None:
    """Days between true onset and the first alarm containing the outbreak areas.

    `runner` performs one day's scan for a given configuration. Returns None if
    the outbreak was never detected within the replay horizon, which is a result
    and must not be silently treated as a large delay."""
    for day in range(outbreak["onset_day"], outbreak["onset_day"] + 60):
        res = runner(cases, scan_day=day, max_days=max_days)
        if res["recurrence_scans"] < threshold:
            continue
        hit = set(res["cluster"]["areas"]) & set(outbreak["areas"])
        if len(hit) >= max(1, len(outbreak["areas"]) // 2):
            return day - outbreak["onset_day"]
    return None


def sweep_max_window(cases, outbreaks, quiet_days, candidates, threshold, runner):
    rows = []
    for mw in candidates:
        delays = [replay_detection_delay(cases, o, mw, threshold, runner) for o in outbreaks]
        detected = [d for d in delays if d is not None]
        alarms = sum(
            runner(cases, scan_day=d, max_days=mw)["recurrence_scans"] >= threshold
            for d in quiet_days)
        rows.append({
            "max_days": mw,
            "detected": f"{len(detected)}/{len(outbreaks)}",
            "median_delay": float(np.median(detected)) if detected else np.nan,
            "alarms_per_year": 365.0 * alarms / len(quiet_days),
        })
        log.info("max_days=%3d: detected %s, median delay %s d, %.1f alarms/yr",
                 mw, rows[-1]["detected"],
                 f"{rows[-1]['median_delay']:.0f}" if detected else "n/a",
                 rows[-1]["alarms_per_year"])
    return pd.DataFrame(rows)

Read the sweep as a two-column trade: detection delay against alarms per year. There is no maximum that optimises both, so the choice is made by asking how many false alarms the response team can absorb in a year and taking the shortest delay available at that budget.

Validation & Edge Cases

1. Never tune on the outbreak you are currently investigating. Selecting a window because it detects the active event is fitting the parameter to the answer. Tune on closed historical events and freeze the configuration before the season starts.

2. Check that the minimum window matches the reporting cadence. On weekly-batched data, a one-day minimum produces cylinders containing a single batch and the scan alarms on the batch:

INFO max_days=  7: detected 1/3, median delay 4 d, 3.2 alarms/yr
INFO max_days= 14: detected 2/3, median delay 6 d, 6.8 alarms/yr
INFO max_days= 28: detected 3/3, median delay 9 d, 12.4 alarms/yr
INFO max_days= 56: detected 3/3, median delay 16 d, 21.7 alarms/yr
WARNING min_days=1 with weekly batching: 78% of alarms have a 1-day window

3. Test sensitivity to the recurrence-interval threshold jointly. Window length and threshold both move the alarm rate, and a sweep over one at a fixed value of the other can suggest a window that is only good at that threshold. Sweep the pair.

4. Consider two configurations rather than a compromise. A short-window configuration for point-source events and a long-window one for diffuse increases cost twice the compute and give both detection profiles. Run them as separate named systems with separate alarm streams, not as one system with two thresholds, so their false-alarm rates stay separately measurable.

5. Re-tune after a surveillance change. A new reporting stream, a testing-policy change, or a case-definition revision all shift the baseline, and a window tuned before the change is tuned for a different system.

6. Tune the spatial and temporal maxima together, not sequentially. They interact: a wide spatial maximum enlarges the cylinder family, which raises the permutation maximum and reduces sensitivity to short windows, so a temporal maximum tuned at one spatial setting is not optimal at another. A coarse grid over both is only a few times the cost of a sweep over one and gives a defensible pair rather than a defensible parameter.

7. Record the sweep’s compute cost alongside its results. A configuration that takes four hours per daily run cannot be operated on a daily cadence, and discovering that after tuning is a wasted sweep. Include runtime as a column in the sweep table so the operating point is chosen from what can actually be run.

8. Prefer a shorter maximum when the outbreak type is unknown. Short-window configurations are more sensitive to the sharp, localised events surveillance systems exist to catch, and diffuse increases are usually visible in aggregate trend monitoring well before a scan would flag them. Where only one configuration can be operated, that asymmetry argues for the shorter window.

9. Re-check the window against the reporting-completeness trim. Trimming five days off the end of the series shortens every evaluated window by the same amount in practice, so a nominal fourteen-day maximum evaluates data ending five days ago. Where the trim is large relative to the window, the effective configuration is not the configured one, and the sweep should be run on the trimmed series to reflect what the system actually does.

The joint sweep over window and threshold is what stops a parameter looking good only at one operating point:

Episodes Per Year Across Window and Threshold A grid of expected false-alarm episodes per year across three maximum window lengths and three recurrence-interval thresholds. At a fourteen-day window the episode rate runs from 8.1 at a threshold of 50 down to 1.2 at 365. At twenty-eight days it runs from 12.4 to 2.0. At fifty-six days from 21.7 to 3.6. Reading either axis alone would suggest a different operating point than reading both. Episodes per year, both axes varied RI ≥ 50 RI ≥ 100 RI ≥ 365 max 14 days 8.1 3.4 1.2 max 28 days 12.4 5.1 2.0 max 56 days 21.7 9.3 3.6 The highlighted column is the operating point a six-episode budget selects

Compliance Notes

  • Record the tuning evidence, not just the parameter. The sweep table is the justification for the chosen window and belongs with the configuration.
  • Freeze the configuration for a stated period and log any change with its date, since an alarm-rate change caused by re-tuning must not be mistaken for an epidemiological change.
  • State the detected-fraction honestly. A window that detected two of three historical outbreaks should be reported that way; converting a non-detection into a large delay flatters the configuration.
  • Keep both configurations’ outputs when running in parallel, so a later review can see which system produced which alert.