When Deep Neural Nets Fail

Adversarial Examples are imperceptible perturbations of natural inputs that induce erroneous predictions. Their corollary: Fooling Images ² — unnatural inputs that trigger high-confidence predictions yet remain unrecognizable to humans.

These two phenomena expose three failures in simple Deep Neural Networks (DNNs): over-confidence, discriminatory narrow-sightedness, and lack of intentionality.

1. Over-confidence - A human would give an out-of-sample input low confidence. The model cannot detect that its frame of reference is incoherent, so it won’t flag the answer as uncertain.

2. Discriminatory Narrow-sightedness - Humans make high-level inferences, partly because we discard much of our visual input; three times per second we go blind from Saccades and don’t notice.

   1. Deep Neural Nets use no such tricks. They dredge through data seeking any pattern with discriminatory power — even features utterly inhuman and unintended. These non-robust features predict well but prove brittle for broader object recognition.

3. Intentionality Failure - These examples aren’t model failures. They’re challenges of intention. Restricting the model to human-interpretable features isn’t the answer — we’d lose valuable, perhaps inhuman, features.

Neural Networks pay far more attention to texture and color than humans do when classifying.

The “Fooling Image” reveals a more fundamental issue:

1. A Lack of Sufficiency - DNNs assume all discriminatory features are sufficient. If the only images with black and yellow stripes are school buses, that becomes a sufficient feature. A human would treat it as a mere indicator, not a conclusion.