Although artificial intelligence (AI) is one of the most important inventions of modern times, it is vulnerable to some age-old problems and learning paradoxes, including the Clever Hans (CH) effect, often observed in nature and replicated in laboratories.
Specifically, a research team led by Technische Universität (TU) Berlin investigated the occurrence of the Clever Hans effect in unsupervised learning of AI models and discovered some surprising results, according to the article in Nature Machine Intelligence published on March 17.
As it happens, Clever Hans was a horse in early 20th century Germany that could supposedly do math, indicating a number by tapping his hoof. However, psychologist Oskar Pfungst discovered that the horse wasn’t actually performing these mental tasks but was observing the involuntary body language of his trainer, who was unaware he was providing any cues – and this became known as the Clever Hans effect.
AI faces the (not so) Clever Hans conundrum
Now, researchers have observed the same thing happening to unsupervised AI models, when models themselves are supposed to recognize patterns and correlations without human directions or a prospect of a reward. By applying customized explainable AI (XAI) techniques to popular representation learning and anomaly detection models for image data, they have realized that the Clever Hans effect is widespread here.
Indeed, unsupervised learning has delivered significant results in modeling the unknown, such as uncovering new cancer subtypes or extracting novel insights from large historical bodies of information, the team pointed out. That said, unsupervised learning models largely suffer from Clever Hans effects.
Implications of the Clever Hans effect in AI
Having said that, a problem appears when a model doesn’t look at the image at all but arrives at conclusions by capturing an insignificant detail – such as the notes near the frame, which the authors of the study could witness themselves when using X-ray images of lungs with and without Covid-19.
They found that, in some cases, the AI didn’t classify the images based on the Covid features but on other characteristics. Specifically, the AI would sometimes include the notes on the edge of the images, leading to a number of incorrect classifications and seriously undermining its usability in medical diagnosis.
In a different test, the researchers used an image of a piece of wood, leading them to a similar conclusion – many unsupervised models latch onto irrelevant patterns like textual annotations in medical X-rays or background artifacts in images to make decisions.
Following the discovery, it becomes clear that there is an urgent need to carefully examine how AI models arrive at their predictions and resolve critical flaws like the Clever Hans effect with proper safeguards, or it could lead to deeply skewed results and misdiagnoses in important real-life areas like medicine.
Additionally, the findings witness how easily AI can be fooled into relying on superficial artifacts rather than learning the true characteristics of defects and represent a warning for various industries that depend on AI for quality control, as unnoticed failures could lead to serious financial losses or safety risks.
Meanwhile, AI is helping humans address many previously difficult challenges in medicine and healthcare, such as designing drugs for issues like inflammatory bowel disease, informing them about calories in their food for more effective weight loss, and more.
ReadNext
