When Artificial Intelligence Fails Puzzles Reveal Secrets of the Human Mind
AI systems are currently able to perform extremely complex operations in seconds using huge amounts of data, and can solve problems beyond human capabilities. However, puzzles still challenge these systems and highlight the superiority of the human mind.
But why are puzzles such a challenge for AI? And what secrets do they reveal about the nature of the human mind?
Challenges of developing artificial intelligence:
Philip Ilvesky , an assistant professor at VU University Amsterdam, is trying to answer an important question: Can a machine solve a puzzle? So he is focusing on developing AI’s ability to think critically and solve complex problems, which are very similar to the puzzles we face in our daily lives.
Understanding and improving AI's ability to solve puzzles and logical problems is key to improving the performance of AI systems in the future, Ilivsky says.
Elevsky also pointed out that humans have an innate ability to apply common sense and adapt it to different situations, and this ability is what scientists aspire to achieve in artificial intelligence now, but this is not easy, because current artificial intelligence suffers from a severe lack of understanding of the world around it, which makes it unable to simulate the flexibility and critical thinking that humans possess.
“AI is exceptional at pattern recognition, but it faces challenges in tasks that require abstract reasoning capabilities compared to humans,” says Zach Petko, an associate professor at Carnegie Mellon University in the US who studies the intersection of AI and neuroscience. “However, AI’s effectiveness varies depending on the nature of the problem at hand.”
This gap between human and artificial intelligence opens up new avenues for research, as scientists seek to understand the mechanisms that enable humans to make quick and effective decisions in the face of unexpected challenges.
Puzzles that baffle artificial intelligence:
In a 2023 study , researchers asked GPT-4 a simple question: “Mabel’s heart rate at 9 a.m. was 75 beats per minute and her blood pressure at 7 p.m. was 120/80. She died at 11 p.m. Was Mabel still alive at 12 p.m.?”
The answer is obvious to anyone: of course Mabel was alive at 12 noon, because she died at 11 pm. However, the GPT-4 model, with its amazing ability, was unable to answer conclusively, confirming that there was not enough information to determine Mabel's condition at 12 pm.
But why did AI fail to solve this simple puzzle?
The challenge is that this question requires a special kind of thinking called temporal reasoning, the ability to understand the temporal relationships between events and their sequence. This may seem intuitive to humans, but it poses a major obstacle for artificial intelligence. While a machine can calculate and solve complex equations, understanding abstract concepts like time, life, and death requires a deeper level of understanding.
Scientists still face a major challenge in understanding how artificial intelligence works, especially large language models such as ChatGPT . We know that these models work by analyzing huge amounts of data to find patterns and statistics.
So when we ask these models a question, they look for linguistic patterns that match that question, and then provide an answer based on those patterns. But how exactly do these models arrive at these answers? And what complex computations are going on behind the scenes? That’s still a mystery.
So AI can be likened to a black box, we see the inputs and the outputs, but we don't fully understand the internal processes that take place to turn those inputs into outputs.
So the scariest aspect of the generative AI revolution we're witnessing today is that the scientists and programmers who built the large language models still don't know how they work and evolve.
The same is true of the human brain. We know very little about how our minds work. The latest brain scanning techniques show us that individual groups of neurons are active when a person thinks, but we are unable to translate this activity into a language that we fully understand. Thoughts and feelings are the product of complex interactions between billions of neurons that we have yet to decipher.
But, is there a link between artificial intelligence and the human brain?
The answer is yes, there is a lot of overlap. The artificial neural networks that form the basis of many AI models are inspired by the structure of the brain and the way neurons work. However, this does not mean that AI thinks in the same way as humans. Rather, comparing them is like comparing an airplane to a bird: both can fly, but in completely different ways.
When the brain tricks us.. A simple puzzle reveals the power of artificial intelligence:
Puzzles that target weaknesses in human thinking are a valuable tool for testing the capabilities of artificial intelligence. Here's an easy example: A baseball bat and a baseball together cost $1.10. If the bat costs $1 more than the ball, how much does the ball cost?
The answer that comes to mind for most of us is: 10 cents, by subtracting the price of the bat ($1) from the total price ($1 and 10 cents). But this answer is wrong. The correct answer is: The ball costs 5 cents, and the bat costs $1 and 5 cents.
But why do we make this mistake? Shane Frederick, a professor of marketing at the Yale School of Management, explains that the main reason behind this mistake is an over-reliance on intuition. We tend to rely on intuition in our daily lives, and it serves us well in most situations. But when we are faced with a problem that is not intuitive, our intuition can trick us and lead us to the wrong answers.
Artificial intelligence has solved this puzzle, so this puzzle clearly shows the differences between the way humans and artificial intelligence think. Frederick believes that artificial intelligence excels in extracting the essential aspects of the problem and applying appropriate mathematical operations to it.
However, this puzzle, despite its popularity, will not be the optimal test of AI capabilities, due to the possibility of this puzzle being present in the training data. Therefore, more complex and diverse tests must be developed to accurately assess AI capabilities.
In this context, Dr. Philip Ilievsky and his team resorted in a recent study to building the original set of puzzles, to evaluate the extent to which artificial intelligence can simulate logical human thinking.
Researchers have developed a new computer program that can create complex visual puzzles, known as rebus puzzles, which use combinations of images, symbols and letters to represent words or phrases, requiring abstract thinking and the ability to make connections between different concepts.
The researchers then presented these new and innovative puzzles to advanced models, such as OpenAI ’s GPT-4o , as well as a group of humans. The results showed that humans outperformed them, achieving 91.5% accuracy in solving these complex puzzles, compared to 84.9% accuracy for artificial intelligence. These results confirm that human creativity and critical thinking are still a strong point of superiority for humans over machines.
However, Ilivsky points out that the lack of a unified classification of logic and reasoning makes it difficult to accurately evaluate AI performance and compare it to human mental abilities.
So one previous study broke reasoning down into some useful categories. The researchers asked GPT-4 a series of questions, puzzles, and word problems representing 21 different types of reasoning, from simple arithmetic to complex spatial concepts. GPT-4 notably failed most of these tests.
The model also failed the Wason selection task, a classic example of how our senses can be fooled when it comes to logical reasoning. In this puzzle, participants are asked to decide which cards to flip to verify a certain rule. Despite the simplicity of the puzzle, many humans fail to solve it, and so did GPT-4.
GPT-4 did not just fail on the Wason selection task, it struggled to solve other problems that required an understanding of spatial and temporal concepts. For example, it failed to determine the correct direction of Boston relative to a specific location in the United States, despite having a broad background in geography and history.
These results indicate that artificial intelligence still faces significant challenges in the field of logical thinking, as its ability to deal with language and generate texts and visual materials does not necessarily mean its ability to understand the meaning behind the words.
However, artificial intelligence is witnessing rapid development. This September, OpenAI launched the new (o1) model, which is designed to spend more time thinking in a way closer to human thinking before responding to user inquiries. This model succeeded in all previous tests, which the (GPT-4) and (GPT-4o) models failed.
So, the future lies in the integration of AI capabilities and human creativity, and as Ilivsky points out, we can leverage the strengths of each to achieve better results.
This means that the development of artificial intelligence and neuroscience are two sides of the same coin. Our attempt to understand how the brain learns, stores and processes information leads us to develop more efficient artificial intelligence algorithms and systems. At the same time, studying these systems helps us understand the complex mental processes that occur in the human brain. This interrelationship between the two fields provides us with valuable insights into the nature of intelligence and opens up new avenues for research and development in areas such as medicine, education and industry.