Zero-Shot Learning and Human Analogical Reasoning!

Author(s): Shravankumar Hiregoudar

Originally published on Towards AI.

Zero-Shot Learning and Human Analogical Reasoning!

Recently, I read David Epstein’s ‘Range’ and stumbled upon Duncker’s Radiation Problem in Chapter 5, where the author talks about analogical problem-solving. As an AI/ML engineer, I was inspired by the striking resemblance between this concept and how ML algorithms tackle zero-shot learning (ZSL) methods. I want to share the lightbulb moments and insights with you here! U+1F4A1

“Analogical thinking takes the new and makes it familiar, or takes the familiar and puts it in a new light, and allows humans to reason through problems they have never seen in unfamiliar contexts. It also allows us to understand that which we cannot see at all.”

The topic of discussion!

1. Zero-Shot, One-Shot, and Few-Shot Prompting
2. Duncker’s Radiation Problem
3. Zero-Shot Learning
4. Conclusion

Zero-Shot, One-Shot, and Few-Shot Prompting

Optimizing prompts for better query outcomes is becoming increasingly important. LLMs, such as ChatGPT, Gemini, and Claude, rely on prompts to generate responses tailored to your inquiries. In short;

Zero-shot prompting is a technique that allows AI models to respond to queries without specific training, leveraging prior knowledge to generate quick and relevant answers. That's like asking a simple question in ChatGPT.

Few-shot prompting refines the model’s performance by providing minimal examples for improved accuracy. (Refer Prompting Guide)

These prompting techniques are rooted in the broader concepts of Zero-Shot, One-Shot, and Few-Shot Learning in machine learning. But, Before diving deeper into the ZSL technique, Let's look at Duncker’s radiation problem!

Duncker’s Radiation Problem

In a 1945 paper, Duncker offered the following scenario: (Research Paper)

Suppose you are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but the patient will die unless the tumor is destroyed. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity, the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities, the rays are harmless to healthy tissue, but they will not affect the tumor, either. What type of procedure might be used to destroy the tumor with the rays and, at the same time, avoid destroying the healthy tissue?

Here’s some help:

A small country was ruled by a strong fortress by a dictator. The fortress was situated in the middle of the country, surrounded by farms and villages. Many roads led to the fortress through the countryside. A rebel general vowed to capture the fortress. The general knew that an attack by his entire army would capture the fortress. He gathered his army at the head of one of the roads, ready to launch a full-scale direct attack. However, the general then learned that the dictator had planted mines on each of the roads. The mines were set so that small bodies of men could pass over them safely since the dictator needed to move his troops and workers to and from the fortress. However, any large force would detonate the mines. It, therefore, seemed impossible to capture the fortress. However, the general devised a simple plan. He divided his armies into small groups and dispatched each group to the head of a different road. When all was ready, he gave the signal, and each group marched down the other road. Each group continued down its road so that the entire army arrived together at the fortress at the same time. In this way, the general captured the fortress and overthrew the dictator.

Any luck on finding a solution?

The answer is eloquent once you take the story into the context of the tumor problem. Divide and conquer! Using the military strategy, you can apply this lesson to the tumor problem. Firing low intensity focused on the tumor allows you to destroy the tumor but leave the surrounding tissue unharmed.

Analogical reasoning is effective in solving Dunker’s radiation problem.

Zero-Shot Learning

In ZSL, the data consists of the following:

1. Seen Classes: These are the data classes used to train the model.
2. Unseen Classes: The trained model needs to classify this data, and this data is not used for training the model.
3. Features: Since no labeled instances belonging to the unseen classes are available, some auxiliary information is necessary to solve the ZSL problem. Such auxiliary information should contain information about all unseen classes, which can be descriptions, semantic information, or word embeddings.

In the above example of how Zero-shot learning works, It extracts (knowledge) attribute features from seen classes such as tiger, horse, and panda during the training period. It predicts zebra, an unseen class, with the help of attribute features (knowledge) from seen classes. This prediction relies on the labeled training data of known classes and understanding how each unknown class is connected to the known ones. (Refer to this article to learn about the in-depth mechanism of ZSL)

Conclusion

Humans have a knack for Zero-Shot Learning thanks to their extensive knowledge base. This base acts like a training ground, giving us a broad understanding of new or unfamiliar categories by linking them to what we already know.

As demonstrated in Duncker’s Radiation Problem example, there’s a fascinating similarity between the ability to identify common traits and the use of analogies to solve new problems. We are connecting the dots between what we know and what we’re learning for the first time. It’s like witnessing Zero-Shot Learning in action!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI