Fresh Data is Fairer Data

My mother’s solution to sibling cake disputes was elegant: one child cuts, the other chooses. My sister and I would cut as evenly as humanly possible; we couldn’t allow the other to get one over on us.

Fairness emerged not from rules about equal slices, but from consequences. Cut unfairly and you got the smaller piece. The cost of your own unfairness landed on you, instantly.

Between siblings, equality was fair; no ‘feature’ of either of us warranted a bigger slice. Most decisions are messier. When a model decides who gets a loan, some features should matter and others shouldn’t. A fair decision, at minimum, is one that doesn’t lean on irrelevant characteristics like race or gender ¹.

Trying to ensure we give models fair exemplars to learn from is tricky, and the two obvious approaches to constructing them both fail.

Remove sensitive attributes like race or gender, and the model finds proxies; postcode, name, and browsing history all correlate with the thing you hid. This is “proxy discrimination”, and it’s near-universal.

Remove correlated attributes too? Uncorrelated features can combine to reconstruct what you removed. The information is encoded redundantly across the data. You can’t scrub it out.

Even deciding which features count as “fair” gets tangled. Take loan repayment. Gender affects income, and income affects whether the borrower repays. Income sits on a causal path from the thing we want to ignore to the thing we want to predict. It is fair to use only so far as its predictive power is its own, and not borrowed from gender upstream. It feels fair because most of that power comes from having money, not from being male. But “most” is doing a lot of work in that sentence, and nobody knows how to do the sum.

Cut and choose

These problems share a root: we’re trying to retrofit fairness onto data produced under unfair conditions. The data reflects the world that generated it, bias² and all.

But moral standards shift, and the record shifts with them. Give BERT the sentence “In 1860, he worked as a ___”, then the same sentence with she, and its guesses for the two sit far apart. Move the year to 2017 and they close much of the gap.

Not every gap closes. Ask which game someone played, or what they did to a bear, and BERT’s he/she difference holds steady or even widens. The model has no sense of progress; it repeats what the text about each decade says.

The world that generates today’s data is imperfect, but certainly fairer than the world that generated yesterday’s.

A society still reckoning with its own biases produces data that reflects that reckoning; people feel the consequences of unfairness and push back. Stale data carries no such correction.³

Today’s consensus has blind spots that future generations will find obvious.

Some are visible already. American regulators’ rule of thumb for discrimination in hiring, the four-fifths rule, lets an employer select one group at four-fifths the rate of another before regulators treat it as evidence of discrimination. Cut a cake five parts to four and the slices come out 56 to 44. My sister would have spotted the difference from across the kitchen. Children police the knife more strictly than the law does. There is plenty of room to improve on the law, and that makes me optimistic.

Fresh data won’t make models fair. But stale data almost guarantees they won’t be.