A new MIT study suggests that artificial intelligence (AI) may sometimes misidentify objects not because it sees things that aren’t there, but because it sees the world in ways that humans cannot.
In previous studies, researchers discovered that while machine learning algorithms can be used to train AI to classify objects, AI will sometimes make odd mistakes if it is presented with slightly modified images. While those images to humans still clearly depict the original object, the slight distortion can cause AI to identify them as very different objects. For instance, in one recent study researchers were able to avoid detection by surveillance algorithms simply by attaching a distorted 2D image to their clothing. Issues of this kind are called adversarial examples.
In an attempt to explain adversarial examples, MIT researchers conducted an experiment to check whether AI is really “hallucinating” in the cases mentioned above. The findings indicate that this is not actually the case, but that AI just detects patterns that humans are not capable of detecting. According to the researchers, this discovery has serious implications for the ability of AI experts to understand the decision-making processes of AI solutions. “If we know that our models are relying on […] microscopic patterns that we don’t see, then we can’t pretend that they are interpretable in a human fashion,” one researcher pointed out.
The problem highlighted by the study resembles a conundrum recognized by AI experts in the context of warfare, where reliance on AI seems bound to complicate the nature of command because military strategy will be increasingly based on AI calculations that no human can understand.
Read more: Artificial Intelligence May Not ‘Hallucinate’ After All.
