Fresh Data is Fairer Data

My mother’s solution to sibling cake disputes was elegant: one child cuts, the other chooses. My sister and I would cut as evenly as humanly possible; we couldn’t allow the other to get one over on us.

Fairness emerged not from rules about equal slices, but from consequences. Cut unfairly and you got the smaller piece. The cost of your own unfairness landed on you, instantly.

Between siblings, equality was fair; no ‘feature’ of either of us warranted a bigger slice. Most decisions are messier. When a model decides who gets a loan, some features should matter and others shouldn’t. A fair decision, at minimum, is one that doesn’t lean on irrelevant characteristics like race or gender ¹.

Trying to ensure we give models fair exemplars to learn from is tricky, and the two obvious approaches to constructing them both fail.

Remove sensitive attributes like race or gender, and the model finds proxies; postcode, name, and browsing history all correlate with the thing you hid. This is “proxy discrimination”, and it’s near-universal.

Remove correlated attributes too? Uncorrelated features can combine to reconstruct what you removed. The information is encoded redundantly across the data. You can’t scrub it out.

Even deciding which features count as “fair” gets tangled. Take loan repayment. Income correlates with gender, but income is also predictive on its own. Is it fair to use? Most of the predictive power comes from having money, not from being male. But “most” is doing a lot of work in that sentence.

Cut and choose

These problems share a root: we’re trying to retrofit fairness onto data produced under unfair conditions. The data reflects the world that generated it, bias² and all.

But moral standards shift. Google’s language models show measurable decreases in gender bias when trained on more recent text.

The world that generates today’s data is, imperfect, but certainly fairer than the world that generated yesterday’s.

This is the cake-cutting principle at scale. A society still reckoning with its own biases produces data that reflects that reckoning; people feel the consequences of unfairness and push back. Stale data carries no such correction.

Today’s consensus has blind spots that future generations will find obvious, much as we look back and wonder how certain things were ever acceptable.

Fresh data won’t make models fair. But stale data almost guarantees they won’t be.