Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Read by thought-leaders and decision-makers around the world. Phone Number: +1-650-246-9381 Email: [email protected]
228 Park Avenue South New York, NY 10003 United States
Website: Publisher: https://towardsai.net/#publisher Diversity Policy: https://towardsai.net/about Ethics Policy: https://towardsai.net/about Masthead: https://towardsai.net/about
Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Founders: Roberto Iriondo, , Job Title: Co-founder and Advisor Works for: Towards AI, Inc. Follow Roberto: X, LinkedIn, GitHub, Google Scholar, Towards AI Profile, Medium, ML@CMU, FreeCodeCamp, Crunchbase, Bloomberg, Roberto Iriondo, Generative AI Lab, Generative AI Lab Denis Piffaretti, Job Title: Co-founder Works for: Towards AI, Inc. Louie Peters, Job Title: Co-founder Works for: Towards AI, Inc. Louis-François Bouchard, Job Title: Co-founder Works for: Towards AI, Inc. Cover:
Towards AI Cover
Logo:
Towards AI Logo
Areas Served: Worldwide Alternate Name: Towards AI, Inc. Alternate Name: Towards AI Co. Alternate Name: towards ai Alternate Name: towardsai Alternate Name: towards.ai Alternate Name: tai Alternate Name: toward ai Alternate Name: toward.ai Alternate Name: Towards AI, Inc. Alternate Name: towardsai.net Alternate Name: pub.towardsai.net
5 stars – based on 497 reviews

Frequently Used, Contextual References

TODO: Remember to copy unique IDs whenever it needs used. i.e., URL: 304b2e42315e

Resources

Unlock the full potential of AI with Building LLMs for Productionβ€”our 470+ page guide to mastering LLMs with practical projects and expert insights!

Publication

Conditional Independence
Artificial Intelligence

Conditional Independence

Last Updated on March 24, 2022 by Editorial Team

Author(s): Ampatishan Sivalingam

Originally published on Towards AI the World’s Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses.

The independence that can be realized in the realΒ world

When it comes to probability theory we all would have heard of joint distribution, marginal distribution, independence, etc. In this article, I will focus my attention on independence, especially conditional independence. If two events A and B are independent if they satisfy the following condition:

In others words, if the happening of event A doesn’t affect the probability of event B happening, both events are said to be independent. From the view of information theory, it can be interpreted as: if knowing A doesn’t provide any additional information about B, then A and B are said to be independent. These are the different interpretations of the concept of independence. There is a huge misunderstanding of thinking of mutually exclusive events as independent events, which is completely wrong. Earlier I also had this misconception, thinking if an event can’t happen if another event has already happened, then they both should be independent. But it's not true, like mentioned above for two events A and B to be independent, no information should be provided when A happens regarding the occurrence of B, but if A and B are mutually exclusive events, i.e A and B can’t occur at the same time. So if we know that A has happened then we are sure that B is not gonna happen, in other words, if we know that A happened it has all the information needed regarding the occurrence of B, which is completely opposite of the definition of independent events. Therefore, mutually exclusive events can never be independent events.

In the real world, realizing (unconditional) independence will be rare, because every event will be most likely influence other events, mostly these influences are directed through an intermediate event. So, if the occurrence of the intermediate event is realized, then there will be no other influence between the other two events, this concept is known as conditional independence, and can be observed in real-world scenarios. The formal definition for conditional independence is asΒ follows:

Now we have dependence as well as conditional independence, now we will look if they two are related in anyΒ way.

  1. Does independence imply conditional independence?

Consider two events A and B to be independent

Now given a event C has occurred, the event A and B has become mutually exclusive event (both can’t happen at the same time), therefore not independent. So independence doesn’t implies conditional independence.

2. Does conditional independence implies independence

Consider we have two coins, one is biased and the other one is a fair coin, if we condition on choosing a fair coin or biased coin, then the coin flips will be independent of each other, on contrast if we don’t condition on a coin, then by looking at the outcomes for a certain period of time we can identify whether we have a biased coin or not, that will give us the information regarding the outcomes of the coin flips from other coin. So conditional independence doesn’t imply independence.


Conditional Independence was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Join thousands of data leaders on the AI newsletter. It’s free, we don’t spam, and we never share your email address. Keep up to date with the latest work in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming aΒ sponsor.

Published via Towards AI

Feedback ↓