Understandability of Deep Learning Models

Author(s): Lalit Kumar

Originally published on Towards AI.

This member-only story is on us. Upgrade to access all of Medium.

Deep learning systems are a kind of black box when it comes to analysing how they give a particular output, and as the size of the model increase this complexity is further increased. These models despite their impressive performance across various domains, often suffer from lack of transparency issue. Their internal workings are very complex and not easy to understand, hence they are sometimes also referred as “black boxes.” This lack of transparency hinders trust and limits their applicability in safety-critical domains. It is difficult to judge how these powerful models arrive at their decisions. This challenge, often referred to as the “deep learning understandability problem,” has spurred significant research efforts to develop techniques that shed light on the inner workings of these models. For, a smaller model, it may be possible to explore the internal representations and try to understand the model's decision-making process. But as the model size increases so is the problem to understand its decision-making mechanism.

Then, how to keep a track of these models functioning and interpret them?

Following are some of the solutions which handle Deep Learning Model’s understandability problem:

This technique… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI