Answers.Body: <blockquote>
    <p>But the conditional probability is clearly not symmetric and captures directed relationships.</p>
  </blockquote>
  
  <p>One needs to consider the kinds of directed relationships that is captured by conditional probability. It surely does capture <em>some</em> kind of association or dependence which could be directed. At the same time, it is not right to say that it <em>surely</em> captures the causal relationships. </p>
  
  <p>Let:</p>
  
  <p>Sun rises = <span class="math-container">$A$</span>, Rooster crows = <span class="math-container">$B$</span>, then, <span class="math-container">$P(A |B)$</span> is bound to be very high but it does not mean that rooster crowing causes sunrise.  </p>
  
  <blockquote>
    <p>How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?</p>
  </blockquote>
  
  <p>He will ask him to go back to school.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-15T22:47:50.547Z
Answers.Text: I do not think that you actually even try to clarify the doubt of the asker. You're just saying that correlation does not imply causation.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T02:56:32.300Z
Answers.Text: @nbro I have tried to correct the argument provided by the asker, which I believe has led to the question in the first place. I think the example in my answer will clarify that conditional probability is not sufficient to explore causal relationships. How to explore causal relationships? That is another question.
Answers.UserDisplayName: naive
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-15T10:27:42.413Z
Answers.LastActivityDate: 2019-08-15T11:11:33.963Z
Answers.LastEditDate: 2019-08-15T11:11:33.963Z
Answers.UserDisplayName: naive
Answers.Body: <p><em>Why isn't conditional probability sufficient to describe causality?</em></p>
  
  <p>Suppose that, when the barometric pressure, in a certain region, drops below a certain level, two things happen</p>
  
  <ol>
  <li><p>the height of the column of mercury in your barometer drops below a certain level</p></li>
  <li><p>a storm occurs</p></li>
  </ol>
  
  <p>We may be tempted to model these relationships with the following graphical model, where each directed edge represents a causal relationship, so, for example, the drop in barometric pressure causes the storm.</p>
  
  <p><a href="https://i.sstatic.net/dmz3j.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/dmz3j.png" alt="enter image description here"></a></p>
  
  <p>However, this graphical (and causal) model is likely wrong (and unintuitive), given that the drop in barometric pressure is likely <a href="https://en.wikipedia.org/wiki/Correlation_does_not_imply_causation" rel="nofollow noreferrer">only correlated with the storm, so it is not the cause of the storm</a>.</p>
  
  <p><em>How can we see that the drop in barometric pressure is or not the cause of the storm?</em></p>
  
  <p>We can compare the probabilities <span class="math-container">$P(A \mid \text{do}(B))$</span> and <span class="math-container">$P(A \mid B)$</span>, where <span class="math-container">$A$</span> is the event "a storm occurs" and <span class="math-container">$B$</span> is the event "drop in barometric pressure". <em>What does <span class="math-container">$\text{do}(B)$</span> mean?</em> It means that we force the event <span class="math-container">$B$</span> to occur, that is, we <strong>force</strong> the drop in barometric pressure to occur. <em>Intuitively, what is then the difference between <span class="math-container">$P(A \mid \text{do}(B))$</span> and <span class="math-container">$P(A \mid B)$</span></em>? In the case of <span class="math-container">$P(A \mid \text{do}(B))$</span>, we force the event <span class="math-container">$B$</span> to always occur. In the case of <span class="math-container">$P(A \mid B)$</span>, we only and passively look at the cases of event <span class="math-container">$A$</span> when event <span class="math-container">$B$</span> occurs (without thus forcing <span class="math-container">$B$</span> to occur). We now know the difference between <span class="math-container">$P(A \mid \text{do}(B))$</span> and <span class="math-container">$P(A \mid B)$</span>. <em>However, how does this help us to understand that <span class="math-container">$B$</span> is not the cause of <span class="math-container">$A$</span>?</em> If <span class="math-container">$B$</span> was a cause of <span class="math-container">$A$</span>, then, if we forced <span class="math-container">$B$</span> always to occur, then the probability of <span class="math-container">$A$</span> should also change accordingly. However, imagine that we are (magically) able to drop the barometric pressure, if the probability of <span class="math-container">$A$</span> does not change accordingly (in this case, if it does not increase), then the storm is not an effect of the drop in barometric pressure.</p>
  
  <p>To conclude, Judea Pearl would say that <strong>do</strong>-operators (or <em>interventions</em>) are required to analyze causal relationships.</p>
  
  <p>The article <a href="https://plato.stanford.edu/entries/causation-probabilistic/#Inte" rel="nofollow noreferrer">Probabilistic Causation</a> by Stanford Encyclopedia of Philosophy gives a good overview of the (probabilistic) causation (or causality) field. In particular, have a look at section 3, which describes causal modeling (according to Pearl). Causal modeling and inference actually involve several nontrivial concepts that require some time to get familiar with, such as interventions (or do-operations), several basic causal relationships (such as <a href="https://en.wikipedia.org/wiki/Causal_model" rel="nofollow noreferrer">forks, chains and colliders</a>), d-separation or Bayesian networks.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T06:45:05.443Z
Answers.Text: So, basically your example hints at performing [controlled experiment](https://en.wikipedia.org/wiki/Scientific_control#Controlled_experiments) to arrive at a conclusion. But, you fail to mention that one needs to control for all other relevant random variables that could possibly affect the occurrence of a storm.
Answers.UserDisplayName: naive
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T09:24:08.873Z
Answers.Text: @naive I didn't want to go into the details of how you implement the do operations or randomized experiments.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T09:59:45.927Z
Answers.Text: Instead of introducing the do-operations, if the answer reflects a bit upon upon controlled experiments it would become more useful for the OP as one can see the OP clearly lacks that understanding. Introducing the do-operation without any background of controlled experiments seems to only provide marginal benefit to the OP.
Answers.UserDisplayName: naive
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T11:08:45.510Z
Answers.Text: @naive I don't think so, because I also explain what do operations are.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-15T22:42:45.840Z
Answers.LastActivityDate: 2019-08-15T22:58:00.943Z
Answers.LastEditDate: 2019-08-15T22:58:00.943Z
Answers.UserDisplayName: nbro
Answers.Body: <h2>About Symmetry</h2>
  
  <p>What Pearl means by <em>being not symmetric</em> is: <span class="math-container">$A=B$</span> and <span class="math-container">$B=A$</span> are exactly identical and lead to the same result in a not-causal scientific framework. For example consider very simple set of equations:</p>
  
  <p><span class="math-container">$$
  \begin{align}
  Z = &amp; \epsilon_z \\
  X = &amp; Z+\epsilon_X \\
  Y = &amp; 2X+Z + \epsilon_y
  \end{align}
  $$</span></p>
  
  <p>From the Algebra point of view, this is a set of 3 equations, and 3 unknowns (consider error terms to be known). You can shuffle the equations, change the LHS and EHS of the equations, add or subtract them. In fact these actions are actually appreciated for solving a linear equation system, righ?! So the system above is identical to this:</p>
  
  <p><span class="math-container">$$
  \begin{align}
  \epsilon_z = &amp; Z \\
  X = &amp; Y - 2Z - \epsilon_Y - \epsilon_X \\
  Y = &amp; 2X+Z + \epsilon_y \\
  \end{align}
  $$</span></p>
  
  <p>But I brought to you this example because the first one is the set of equations for a very fundamental causal structure called <em>confounding</em> or <em>common cause</em> structure, Where <span class="math-container">$X$</span> is the exposure, or treatment, <span class="math-container">$Y$</span> is the outcome, and <span class="math-container">$Z$</span> is the parent of both <span class="math-container">$X$</span> and <span class="math-container">$Y$</span>. The orders of the equations and the RHS/LHS variables in this <strong>Structural Equation Models</strong> actually mean something. First you calculate <span class="math-container">$Z$</span>, then go for <span class="math-container">$X$</span>, then calculate <span class="math-container">$Y$</span>. In this case, the second system is pointing to a completely different causal structure (or to be honest, it is not even look like a legit structural equation)</p>
  
  <h2>About sufficiency of Probability Theory for Causal Inference</h2>
  
  <p>First I would like to say this would be a very ironic question to ask from Pearl, as he also mentioned in one of his interviews, because he has had a significant contribution to the realm of probability theory with <em>bayesian network</em> and bayesian inference framework! And now it is like he is against himself. But he is for a good reason.</p>
  
  <p><em>why Probability is not enough</em> can be and has been answered with formal proofs, equations and explanations. But There are a lot of examples that will draw your attention to this truth. I recommend you to read about <em>Simpson's Paradox</em>. It will be a great example to see how probability theory is incapable of causal inference.</p>
  
  <p>The fact that probability is not enough, only resembles the idea that <em>correlation is not necessarily causation</em>. and that is true. Again, read about <em>spurious correlations</em> and you will get it. Just think of this funny example: During the year, Crime rate and Ice cream sales amount are highly correlated, THUS we must ban ice cream sales to control the crime rate. and the problem with this dumb inference is that we have not taken into account the common cause which is <em>heat</em> or <em>summer</em> that accounts for the perceived correlation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T17:10:08.853Z
Answers.LastActivityDate: 2020-04-21T17:10:08.853Z
Answers.UserDisplayName: Alireza
Answers.Body: <p>Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:</p>
  
  <p><span class="math-container">$$
  P(A|B) = \frac{P(B|A)P(A)}{P(B)}
  $$</span></p>
  
  <p>So if you have <span class="math-container">$P(B|A)$</span>, <span class="math-container">$P(A)$</span>, and <span class="math-container">$P(B)$</span>, we can determine <span class="math-container">$P(A|B)$</span>, and similarly you can determine <span class="math-container">$P(B|A)$</span> from <span class="math-container">$P(A|B)$</span>, <span class="math-container">$P(B)$</span> and <span class="math-container">$P(A)$</span>. Just by looking at <span class="math-container">$P(B|A)$</span> and <span class="math-container">$P(A|B)$</span>, it is therefore impossible to tell what the causal direction is (if any).</p>
  
  <p>In fact, probabilistic inference usually works the other way round: When there is a <em>known</em> causal relation, say from <em>diseases</em> <span class="math-container">$A$</span> to <em>symptoms</em> <span class="math-container">$B$</span>, we usually have <span class="math-container">$P(B|A)$</span>, and are interested in the diagnostic reasoning task of determining <span class="math-container">$P(A|B)$</span> from that. (The only other thing we need for that is the prior probability <span class="math-container">$P(A)$</span> since <span class="math-container">$P(B)$</span> is just a normalization factor.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-15T23:50:27.663Z
Answers.LastActivityDate: 2019-08-15T23:50:27.663Z
Answers.UserDisplayName: Jens Classen
Body: <p>I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships. </p>
  
  <p>How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?</p>
  
Tags: <philosophy><comparison><conditional-probability><causation>
Title: Why isn't conditional probability sufficient to describe causality?
UserDisplayName: user3180
ThreadText: # Why isn't conditional probability sufficient to describe causality?
  
  (Asked by: username_0 on 2019-08-15)
  
  I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships.
  
  How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?
  
  **Tags:** philosophy, comparison, conditional-probability, causation
  
  ---
  
  ## 4 answers
  
  ### Answer by username_4 on 2019-08-15. Score: 6 | Accepted answer
  
  Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:
  
  $$
  P(A|B) = \frac{P(B|A)P(A)}{P(B)}
  $$
  
  So if you have $P(B|A)$, $P(A)$, and $P(B)$, we can determine $P(A|B)$, and similarly you can determine $P(B|A)$ from $P(A|B)$, $P(B)$ and $P(A)$. Just by looking at $P(B|A)$ and $P(A|B)$, it is therefore impossible to tell what the causal direction is (if any).
  
  In fact, probabilistic inference usually works the other way round: When there is a *known* causal relation, say from *diseases* $A$ to *symptoms* $B$, we usually have $P(B|A)$, and are interested in the diagnostic reasoning task of determining $P(A|B)$ from that. (The only other thing we need for that is the prior probability $P(A)$ since $P(B)$ is just a normalization factor.)
  
  
  ---
  
  ### Answer by username_1 on 2019-08-15. Score: 3
  
  > But the conditional probability is clearly not symmetric and captures directed relationships.
  
  One needs to consider the kinds of directed relationships that is captured by conditional probability. It surely does capture *some* kind of association or dependence which could be directed. At the same time, it is not right to say that it *surely* captures the causal relationships.
  
  Let:
  
  Sun rises = $A$, Rooster crows = $B$, then, $P(A |B)$ is bound to be very high but it does not mean that rooster crowing causes sunrise.
  
  > How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?
  
  He will ask him to go back to school.
  
  > **Comments:**
  > **username_2**: I do not think that you actually even try to clarify the doubt of the asker. You're just saying that correlation does not imply causation.  
  > **username_1**: @nbro I have tried to correct the argument provided by the asker, which I believe has led to the question in the first place. I think the example in my answer will clarify that conditional probability is not sufficient to explore causal relationships. How to explore causal relationships? That is another question.  
  
  ---
  
  ### Answer by username_2 on 2019-08-15. Score: 0
  
  *Why isn't conditional probability sufficient to describe causality?*
  
  Suppose that, when the barometric pressure, in a certain region, drops below a certain level, two things happen
  
  1. the height of the column of mercury in your barometer drops below a certain level
  2. a storm occurs
  
  We may be tempted to model these relationships with the following graphical model, where each directed edge represents a causal relationship, so, for example, the drop in barometric pressure causes the storm.
  
  [![enter image description here](https://i.sstatic.net/dmz3j.png)](https://i.sstatic.net/dmz3j.png)
  
  However, this graphical (and causal) model is likely wrong (and unintuitive), given that the drop in barometric pressure is likely [only correlated with the storm, so it is not the cause of the storm](https://en.wikipedia.org/wiki/Correlation_does_not_imply_causation).
  
  *How can we see that the drop in barometric pressure is or not the cause of the storm?*
  
  We can compare the probabilities $P(A \mid \text{do}(B))$ and $P(A \mid B)$, where $A$ is the event "a storm occurs" and $B$ is the event "drop in barometric pressure". *What does $\text{do}(B)$ mean?* It means that we force the event $B$ to occur, that is, we **force** the drop in barometric pressure to occur. *Intuitively, what is then the difference between $P(A \mid \text{do}(B))$ and $P(A \mid B)$*? In the case of $P(A \mid \text{do}(B))$, we force the event $B$ to always occur. In the case of $P(A \mid B)$, we only and passively look at the cases of event $A$ when event $B$ occurs (without thus forcing $B$ to occur). We now know the difference between $P(A \mid \text{do}(B))$ and $P(A \mid B)$. *However, how does this help us to understand that $B$ is not the cause of $A$?* If $B$ was a cause of $A$, then, if we forced $B$ always to occur, then the probability of $A$ should also change accordingly. However, imagine that we are (magically) able to drop the barometric pressure, if the probability of $A$ does not change accordingly (in this case, if it does not increase), then the storm is not an effect of the drop in barometric pressure.
  
  To conclude, Judea Pearl would say that **do**-operators (or *interventions*) are required to analyze causal relationships.
  
  The article [Probabilistic Causation](https://plato.stanford.edu/entries/causation-probabilistic/#Inte) by Stanford Encyclopedia of Philosophy gives a good overview of the (probabilistic) causation (or causality) field. In particular, have a look at section 3, which describes causal modeling (according to Pearl). Causal modeling and inference actually involve several nontrivial concepts that require some time to get familiar with, such as interventions (or do-operations), several basic causal relationships (such as [forks, chains and colliders](https://en.wikipedia.org/wiki/Causal_model)), d-separation or Bayesian networks.
  
  > **Comments:**
  > **username_1**: So, basically your example hints at performing [controlled experiment](https://en.wikipedia.org/wiki/Scientific_control#Controlled_experiments) to arrive at a conclusion. But, you fail to mention that one needs to control for all other relevant random variables that could possibly affect the occurrence of a storm.  
  > **username_2**: @naive I didn't want to go into the details of how you implement the do operations or randomized experiments.  
  > **username_1**: Instead of introducing the do-operations, if the answer reflects a bit upon upon controlled experiments it would become more useful for the OP as one can see the OP clearly lacks that understanding. Introducing the do-operation without any background of controlled experiments seems to only provide marginal benefit to the OP.  
  > **username_2**: @naive I don't think so, because I also explain what do operations are.  
  
  ---
  
  ### Answer by username_3 on 2020-04-21. Score: 0
  
  ## About Symmetry
  
  What Pearl means by *being not symmetric* is: $A=B$ and $B=A$ are exactly identical and lead to the same result in a not-causal scientific framework. For example consider very simple set of equations:
  
  $$
  \begin{align}
  Z = & \epsilon\_z \\
  X = & Z+\epsilon\_X \\
  Y = & 2X+Z + \epsilon\_y
  \end{align}
  $$
  
  From the Algebra point of view, this is a set of 3 equations, and 3 unknowns (consider error terms to be known). You can shuffle the equations, change the LHS and EHS of the equations, add or subtract them. In fact these actions are actually appreciated for solving a linear equation system, righ?! So the system above is identical to this:
  
  $$
  \begin{align}
  \epsilon\_z = & Z \\
  X = & Y - 2Z - \epsilon\_Y - \epsilon\_X \\
  Y = & 2X+Z + \epsilon\_y \\
  \end{align}
  $$
  
  But I brought to you this example because the first one is the set of equations for a very fundamental causal structure called *confounding* or *common cause* structure, Where $X$ is the exposure, or treatment, $Y$ is the outcome, and $Z$ is the parent of both $X$ and $Y$. The orders of the equations and the RHS/LHS variables in this **Structural Equation Models** actually mean something. First you calculate $Z$, then go for $X$, then calculate $Y$. In this case, the second system is pointing to a completely different causal structure (or to be honest, it is not even look like a legit structural equation)
  
  ## About sufficiency of Probability Theory for Causal Inference
  
  First I would like to say this would be a very ironic question to ask from Pearl, as he also mentioned in one of his interviews, because he has had a significant contribution to the realm of probability theory with *bayesian network* and bayesian inference framework! And now it is like he is against himself. But he is for a good reason.
  
  *why Probability is not enough* can be and has been answered with formal proofs, equations and explanations. But There are a lot of examples that will draw your attention to this truth. I recommend you to read about *Simpson's Paradox*. It will be a great example to see how probability theory is incapable of causal inference.
  
  The fact that probability is not enough, only resembles the idea that *correlation is not necessarily causation*. and that is true. Again, read about *spurious correlations* and you will get it. Just think of this funny example: During the year, Crime rate and Ice cream sales amount are highly correlated, THUS we must ban ice cream sales to control the crime rate. and the problem with this dumb inference is that we have not taken into account the common cause which is *heat* or *summer* that accounts for the perceived correlation.
  
  
  ---
  
  