Spatial Epi Public Health GIS

Complementary Suppression to Defeat Back-Calculation

This guide, part of Dasymetric Suppression and Small-Count Disclosure Control, covers the second half of cell suppression: hiding enough other cells that the small ones cannot be recovered from the totals.

Problem Context & Constraints

A table of cases by county and age group shows a 3 in one cell. Policy requires suppressing counts below 11, so the 3 becomes a dash. But the table also shows the county’s total across ages and the age group’s total across counties. Subtract the visible cells from either total and the 3 reappears. Primary suppression alone protects nothing when margins are published.

Complementary suppression hides additional cells — the complements — so that every primary cell can no longer be pinned down. The minimum is a second suppressed cell in the same row and in the same column, but a single complement is often not enough: if a row has two suppressed cells and one of them is also alone in its column, the column total recovers it, and then the row total recovers the other. The suppression pattern must be protected as a system.

Spatial tables add hierarchy. County counts sum to region counts, which sum to the state. A suppressed county in a region with one other county is recovered from the region total unless the other is also suppressed. Tract tables nest in counties; ZIP code tables cross them. The complements have to protect the primary cells against every published total, at every level.

Two goals compete. Suppress enough to protect every primary cell to a required margin, and suppress as little as possible, measured by the number of cells or, better, the number of cases hidden.

Why One Dash Is Not Enough Two small tables with row and column totals. In the left table only the primary cell with value three is suppressed; the row total minus the visible cells recovers it exactly. In the right table, three complementary cells are also suppressed so that each suppressed cell shares its row and its column with another suppressed cell; the totals now allow a range of values for each, and the primary cell cannot be pinned down. Primary only versus primary plus complements Primary only 24 — 31 58 40 17 22 79 58 − 24 − 31 = 3 recovered With complements 24 — — 58 40 — — 79 primary could be 0 to 20 protected every suppressed cell needs a suppressed partner in each published total it belongs to

Prerequisites

  • python 3.11
  • numpy 1.26.4, pandas 2.2.2, pulp 2.8.0 with CBC
  • The unsuppressed table with all published totals, including geographic roll-ups
  • The program’s primary suppression rule and protection margin from its policy file
  • A hierarchy file listing which cells sum to which totals

Step-by-Step Solution

Express the table and all its totals as linear constraints, choose complements with an integer program that minimises suppressed cases while giving each primary cell a feasible range at least as wide as the protection margin, then verify each primary cell’s range with a pair of linear programs.

# Complementary suppression by integer programming, verified with interval bounds.
# Pinned: numpy==1.26.4, pandas==2.2.2, pulp==2.8.0
import logging

import numpy as np
import pandas as pd
import pulp

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("complementary")

K = 11            # primary threshold
MARGIN = 0.5      # each primary must be uncertain by at least MARGIN * K either way


def totals(cells: pd.DataFrame, hierarchy: list[list[int]]) -> list[list[int]]:
    """hierarchy: lists of cell indices that sum to a published total
    (rows, columns and geographic roll-ups)."""
    return [grp for grp in hierarchy if len(grp) > 1]


def interval(cells, groups, suppressed: set[int], target: int) -> tuple[float, float]:
    """Min and max value the target cell can take given published values and totals."""
    bounds = []
    for sense in (pulp.LpMinimize, pulp.LpMaximize):
        m = pulp.LpProblem("bound", sense)
        x = {i: pulp.LpVariable(f"x{i}", lowBound=0) for i in suppressed}
        m += x[target]
        for grp in groups:
            hidden = [i for i in grp if i in suppressed]
            if not hidden:
                continue
            shown = sum(cells.at[i, "count"] for i in grp if i not in suppressed)
            m += pulp.lpSum(x[i] for i in hidden) == cells.loc[grp, "count"].sum() - shown
        m.solve(pulp.PULP_CBC_CMD(msg=False))
        bounds.append(x[target].value())
    return bounds[0], bounds[1]


def choose_complements(cells: pd.DataFrame, groups, max_rounds: int = 50) -> set[int]:
    """Greedy-plus-verification: add the cheapest complement that widens the tightest
    primary's interval, until every primary meets the margin."""
    primaries = set(cells.index[(cells["count"] > 0) & (cells["count"] < K)])
    suppressed = set(primaries)
    for rnd in range(max_rounds):
        weak = []
        for p in primaries:
            lo, hi = interval(cells, groups, suppressed, p)
            v = cells.at[p, "count"]
            if v - lo < MARGIN * K or hi - v < MARGIN * K:
                weak.append((min(v - lo, hi - v), p))
        if not weak:
            log.info("round %d: all %d primaries protected with %d complements",
                     rnd, len(primaries), len(suppressed - primaries))
            return suppressed
        _, p = min(weak)
        partners = {i for grp in groups if p in grp for i in grp} - suppressed
        best = min(partners, key=lambda i: cells.at[i, "count"])
        suppressed.add(best)
    raise RuntimeError("complement search did not converge")

The loop is a greedy heuristic with exact verification. At each round it finds the primary cell whose feasible interval is tightest, adds as a complement the smallest-count cell that shares a published total with it, and recomputes all intervals. It stops when every primary cell’s value could lie anywhere within the protection margin on either side. Choosing the smallest partner minimises the number of cases hidden, which keeps the published table as informative as possible.

Exact optimisation is possible — an integer program over which cells to suppress, with the interval constraints linearised — but is expensive for large tables. The greedy approach with verification gives a safe pattern that is usually close to minimal, and verification means safety never depends on the heuristic being clever.

Protecting Against Every Level of Total A three-level hierarchy. The state total sums two regions. Region one sums three counties; region two sums two counties. A suppressed county in region two, which has only two counties, is recovered from the region total unless its sibling county is also suppressed, which in turn must be protected from the state total by a suppression in region one. The complement search follows these links across levels. Suppressions must hold at every level State Region 1 Region 2 County A B: comp. County C D: 4 E: comp. D needs E within region 2; region 2's hidden total then needs B within the state

Validation & Edge Cases

Zeros. A published zero is exact information. If a primary cell’s row contains zeros, they tighten the bounds for the others. Treat structural zeros (impossible combinations) as known, and consider whether sampling zeros in small areas should themselves be protected.

Singleton rows. A row with only one non-zero cell equals its total, so suppressing the cell without suppressing the total is useless. Suppress the total too, or merge the row with a neighbour before publishing.

Multiple tables sharing totals. Two tables published from the same data — cases by county and age, cases by county and sex — share county totals. Complements chosen independently for each can leave the county total recoverable from one table and used against the other. Solve them jointly, with the shared totals in both constraint sets, or run the differencing check in Detecting Differencing Attacks Across Published Tables on the pair.

Information loss. Report the number of cells and cases suppressed, as a share of the table. If complements hide more than a small fraction of cases, consider coarser geography or broader categories instead; a table that is 40 percent dashes serves nobody.

Consistency over time. Suppression patterns that change from month to month let readers difference consecutive tables. Where a table is published regularly, prefer patterns that are stable over time — for example, always suppressing the same small counties — even at some cost in information.

Rates and maps. Suppressing a count must also suppress its rate, and a choropleth must show suppressed areas as suppressed, not as zero or as a colour derived from an unsuppressed rate. Check the map export, not just the table.

Suppressed cells that look like zeros. Some downstream tools read an empty cell or a dash as zero when the table is loaded, and then compute totals that silently differ from the published ones. Use an explicit suppression marker defined in the data dictionary, and test that the published file loads with those cells as missing rather than zero in the tools readers are likely to use.

INFO complementary round 7: all 23 primaries protected with 19 complements
INFO complementary suppressed cells=42/612 (6.9%) cases hidden=318/14880 (2.1%)
Verifying the Pattern Horizontal intervals for four primary cells show the range of values each could take given the published totals. Each true value sits inside its interval. A dashed band marks the required margin of five and a half on either side. All four intervals extend beyond the band on both sides, so the pattern passes. An interval that ended inside the band would trigger another complement. Each primary's feasible range must exceed the margin primary 1 primary 2 primary 3 primary 4 dot: true value bar: feasible range a range ending within the margin triggers another complement

Compliance Notes

Record the primary threshold, protection margin, hierarchy of totals, the primary and complementary cells, each primary’s verified interval, and the information loss. Keep the unsuppressed table and the intervals inside the secure environment; they reveal exactly what the suppression hides.

Publish a note with the table explaining that some cells are suppressed to protect small counts and that others are suppressed so the small ones cannot be calculated. Readers who understand why a cell with 180 cases is hidden are less likely to request it, and less likely to assume an error.