Answers.Body: <p>The <span class="math-container">$MI$</span> is easy enough.
  For <span class="math-container">$MI(X \to Z)$</span>, we get <span class="math-container">$U([0.5, 0.5]) - 0.5 U([0.9, 0.1]) - 0.5 U([0.1, 0.9])$</span>. The <span class="math-container">$[0.5, 0.5]$</span> comes from <span class="math-container">$0.5 [0.9, 0.1] + 0.5[0.1, 0.9]$</span> -- this is how to calculate <span class="math-container">$P(Z)$</span> from <span class="math-container">$P(Z|X)$</span> and <span class="math-container">$P(X)$</span>.
  For <span class="math-container">$MI(X \to Y)$</span>, we need to marginalize out <span class="math-container">$Z$</span>. This takes a bit of work, but is not too hard. I manually verified all of the reported values in the figure, and they are all correct.</p>
  <p>The <span class="math-container">$\mathrm{LS}{\mathrm{true}}$</span> values are a bit trickier, but still feasible. I managed to get the same values. It is important when calculating <span class="math-container">$\mathrm{LS}{\mathrm{true}}(X \to Z)$</span> that <span class="math-container">$Y$</span> is not a parent, so it does not come into it -- which is why the MI gives exactly the same value.</p>
  <p>The &quot;blind&quot; version took me a bit more effort to implement, but again, I am getting the same values as reported in the table.
  I used the &quot;simple formula&quot; at the top of page 5.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-27T04:39:56.650Z
Answers.Text: Thank you very much @Robby, it helps me understand some part. I don't have a strong mathematical background, which makes me struggle to translate the formulas and numbers in the paper. I'm doing a python code for this formula, and still figure out which variables need to hold which numbers. So please apologize.
  
  When you mentioned $MI(X \to Z)$ = $U([0.5,0.5])−0.5U([0.9,0.1])−0.5U([0.1,0.9])$, it does use formula (2) $U(Z)-U(Z|X)$, right? 
  There is also a formula 
  $\sum\limits_{x,y}P(x,y) log_{2} (\frac{P(x,y)}{P(x)P(y)})$, and I'm not sure how to derive numbers for $P(x,y), P(x), P(y)$.
Answers.UserDisplayName: qillbel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-27T04:48:04.933Z
Answers.Text: Yes, I used formula (2) for that.
  
  $P(x,y) = P(y | x) \cdot P(x)$. You might want to take an introduction to probability theory, make sure it covers things up to conditional probabilities, entropy and Bayes' theorem; you'll encounter those a lot.
Answers.UserDisplayName: Robby Goetschalckx
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-27T05:17:49.120Z
Answers.Text: Thanks @Robby,
  
  I will look at those materials asap. I have quickly tried to find the numbers for: $P(z,x)=0.5; P(x)=0,5; P(z)=0.5$. 
  
  I got it from $P(z,x)$ = the average of $P(z|x=true)$ and $P(z|x=false)$, which yield 0.5. $P(x) = P(x=true) = 0.5$. $P(z=true)$ = the average of 0.9 and 0.1 = 0.5. But the result is different from the table when I plug in those number in the formula. 
  
  I believe I calculated those wrongly, but it will help me to also understand the materials that you suggested if I know where my mistakes are. Thanks again Robby and I will update my understanding.
Answers.UserDisplayName: qillbel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-27T14:38:35.717Z
Answers.Text: If $P(z|x) = 0.5$ and $P(x) = 0.5$, then $P(x, z) = 0.5 \cdot 0.5 = 0.25$.
Answers.UserDisplayName: Robby Goetschalckx
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-27T14:49:26.503Z
Answers.Text: The calculation you did ($P(z=\mathrm{True}|x=\mathrm{True}) P(x=\mathrm{True}) + P(z =\mathrm{True}| x=\mathrm{False})$ gives us $P(z=\mathrm{True})$, not $P(z, x)$.
Answers.UserDisplayName: Robby Goetschalckx
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-31T18:55:52.517Z
Answers.Text: Hey @Robby, 
  
  I have done the calculation for $(X \to Y)$ and $(Z \to Y)$.
  
  $(X \to Y) = U([0.4982, 0.5017]) - 0.5([0.41, 0.09]) - 0.5([0.094, 0.406])$ = 0.1556174 
  $(Z \to Y) = U([0.4982, 0.5017]) - 0.5([0.4495, 0.0505]) - 0.5([0.0545, 0.4455])$ = 0.2576914
  
  As you see, my results are half of the correct results and I'm wondering which my mistakes are.
  
  Some details of my calculations are:
  $U(Y)= 0.9999538$ 
  $U(Y|X)= 0.844336$ 
  $U(Y|Z)= 0.7422624$
Answers.UserDisplayName: qillbel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-31T19:06:37.567Z
Answers.Text: The Joint distribution:
  $P(Y=t | X=t,Z=t) * P(Z=t | X=t) * P(X=t) =0.405$
  $P(Y=t | X=f,Z=t) * P(Z=t | X=f) * P(X=f) =0.0445$
  $P(Y=t | X=t,Z=f) * P(Z=f | X=t) * P(X=t) =0.005$
  $P(Y=t | X=f,Z=f) * P(Z=f | X=f) * P(X=f) =0.0495$
  $P(Y=f | X=t,Z=t) * P(Z=t | X=t) * P(X=t) =0.045$
  $P(Y=f | X=f,Z=t) * P(Z=t | X=f) * P(X=f) =0.0055$
  $P(Y=f | X=t,Z=f) * P(Z=f | X=t) * P(X=t) =0.045$
  $P(Y=f | X=f,Z=f) * P(Z=f | X=f) * P(X=f) =0.4005$
  
  The Marginal:
  $P(Z=t) = 0.5$ and $P(Z=f) = 0.5$
  $P(Y=t) = 0.504$ and $P(Y=f) = 0.496$
  
  I'm sorry I cannot present the comment with a clear formating. Thanks
Answers.UserDisplayName: qillbel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-31T19:38:27.523Z
Answers.Text: Your joint and marginal distributions are correct. U(Y) is correct as well. For U(Y|X), that is equal to U(X, Y) - U(X), which gives me 0.68867. I think that's where your calculations are off.
Answers.UserDisplayName: Robby Goetschalckx
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-01T05:35:14.757Z
Answers.Text: Thanks @Robby,
  
  I calculated $U(X,Y)$, but, it seems even off from the result:
  1.) Find $P(X,Y)$ by marginalizing $Z$. This gives me $P(X,Y)=0.494$.
  2.) Apply $0.494 * log_{2} \frac{1}{0.494}$. This gives me $U(X,Y) = 0.5026$.
  3.) Then $U(Y|X)=U(X,Y)-U(X)=(-4973959)$.
  
  Previously, I calculated $U(Y|X)$, as followed:
  1.) Calculate $P(X=t) * U(Y|X=t) = 0.5*0.8400385= 0.42002$.
  2.) Calculate $P(X=f) * U(Y|X=f) = 0.5*0.8486344= 0.42432$.
  3.) Then $U(Y|X)=0.84434$.
  4.) Finally, $MI(X,Y) = U(Y) - U(Y|X) = 0.9999538 - 0.84434 = 0.15562$, which half of the correct result.
  
  Thanks so much.
Answers.UserDisplayName: qillbel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-26T18:10:50.180Z
Answers.LastActivityDate: 2020-08-26T18:10:50.180Z
Answers.UserDisplayName: Robby Goetschalckx
Body: <p>I'm trying to understand how to calculate the strength of every arc in a Bayesian Network.</p>
  <p>I came across this report <a href="https://smartech.gatech.edu/bitstream/handle/1853/14331/GT-IIC-07-01.pdf?sequence=1&amp;isAllowed=y" rel="nofollow noreferrer">Measuring Connection Strengths and Link Strengths in Discrete Bayesian Networks</a>, but I got lost in the calculation.</p>
  <p>In particular, how are the values of Link Strength true, Link Strength blind, and Mutual Information computed in Table 1?</p>
  <p><a href="https://i.sstatic.net/UMRTd.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UMRTd.png" alt="enter image description here" /></a></p>
  
Tags: <papers><bayesian-networks>
Title: How are the "Link Strength true", "Link Strength blind" and "Mutual Information" calculated in this report on Bayesian networks?
UserDisplayName: qillbel
ThreadText: # How are the "Link Strength true", "Link Strength blind" and "Mutual Information" calculated in this report on Bayesian networks?
  
  (Asked by: username_0 on 2020-08-26)
  
  I'm trying to understand how to calculate the strength of every arc in a Bayesian Network.
  
  I came across this report [Measuring Connection Strengths and Link Strengths in Discrete Bayesian Networks](https://smartech.gatech.edu/bitstream/handle/1853/14331/GT-IIC-07-01.pdf?sequence=1&isAllowed=y), but I got lost in the calculation.
  
  In particular, how are the values of Link Strength true, Link Strength blind, and Mutual Information computed in Table 1?
  
  [![enter image description here](https://i.sstatic.net/UMRTd.png)](https://i.sstatic.net/UMRTd.png)
  
  **Tags:** papers, bayesian-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-08-26. Score: 0
  
  The $MI$ is easy enough.
  For $MI(X \to Z)$, we get $U([0.5, 0.5]) - 0.5 U([0.9, 0.1]) - 0.5 U([0.1, 0.9])$. The $[0.5, 0.5]$ comes from $0.5 [0.9, 0.1] + 0.5[0.1, 0.9]$ -- this is how to calculate $P(Z)$ from $P(Z|X)$ and $P(X)$.
  For $MI(X \to Y)$, we need to marginalize out $Z$. This takes a bit of work, but is not too hard. I manually verified all of the reported values in the figure, and they are all correct.
  
  The $\mathrm{LS}{\mathrm{true}}$ values are a bit trickier, but still feasible. I managed to get the same values. It is important when calculating $\mathrm{LS}{\mathrm{true}}(X \to Z)$ that $Y$ is not a parent, so it does not come into it -- which is why the MI gives exactly the same value.
  
  The "blind" version took me a bit more effort to implement, but again, I am getting the same values as reported in the table.
  I used the "simple formula" at the top of page 5.
  
  > **Comments:**
  > **username_0**: Thank you very much @username_1, it helps me understand some part. I don't have a strong mathematical background, which makes me struggle to translate the formulas and numbers in the paper. I'm doing a python code for this formula, and still figure out which variables need to hold which numbers. So please apologize.  When you mentioned $MI(X \to Z)$ = $U([0.5,0.5])−0.5U([0.9,0.1])−0.5U([0.1,0.9])$, it does use formula (2) $U(Z)-U(Z|X)$, right?  There is also a formula  $\sum\limits_{x,y}P(x,y) log_{2} (\frac{P(x,y)}{P(x)P(y)})$, and I'm not sure how to derive numbers for $P(x,y), P(x), P(y)$.  
  > **username_1**: Yes, I used formula (2) for that.  $P(x,y) = P(y | x) \cdot P(x)$. You might want to take an introduction to probability theory, make sure it covers things up to conditional probabilities, entropy and Bayes' theorem; you'll encounter those a lot.  
  > **username_0**: Thanks @username_1,  I will look at those materials asap. I have quickly tried to find the numbers for: $P(z,x)=0.5; P(x)=0,5; P(z)=0.5$.   I got it from $P(z,x)$ = the average of $P(z|x=true)$ and $P(z|x=false)$, which yield 0.5. $P(x) = P(x=true) = 0.5$. $P(z=true)$ = the average of 0.9 and 0.1 = 0.5. But the result is different from the table when I plug in those number in the formula.   I believe I calculated those wrongly, but it will help me to also understand the materials that you suggested if I know where my mistakes are. Thanks again username_1 and I will update my understanding.  
  > **username_1**: If $P(z|x) = 0.5$ and $P(x) = 0.5$, then $P(x, z) = 0.5 \cdot 0.5 = 0.25$.  
  > **username_1**: The calculation you did ($P(z=\mathrm{True}|x=\mathrm{True}) P(x=\mathrm{True}) + P(z =\mathrm{True}| x=\mathrm{False})$ gives us $P(z=\mathrm{True})$, not $P(z, x)$.  
  > **username_0**: Hey @username_1,   I have done the calculation for $(X \to Y)$ and $(Z \to Y)$.  $(X \to Y) = U([0.4982, 0.5017]) - 0.5([0.41, 0.09]) - 0.5([0.094, 0.406])$ = 0.1556174  $(Z \to Y) = U([0.4982, 0.5017]) - 0.5([0.4495, 0.0505]) - 0.5([0.0545, 0.4455])$ = 0.2576914  As you see, my results are half of the correct results and I'm wondering which my mistakes are.  Some details of my calculations are: $U(Y)= 0.9999538$  $U(Y|X)= 0.844336$  $U(Y|Z)= 0.7422624$  
  > **username_0**: The Joint distribution: $P(Y=t | X=t,Z=t) * P(Z=t | X=t) * P(X=t) =0.405$ $P(Y=t | X=f,Z=t) * P(Z=t | X=f) * P(X=f) =0.0445$ $P(Y=t | X=t,Z=f) * P(Z=f | X=t) * P(X=t) =0.005$ $P(Y=t | X=f,Z=f) * P(Z=f | X=f) * P(X=f) =0.0495$ $P(Y=f | X=t,Z=t) * P(Z=t | X=t) * P(X=t) =0.045$ $P(Y=f | X=f,Z=t) * P(Z=t | X=f) * P(X=f) =0.0055$ $P(Y=f | X=t,Z=f) * P(Z=f | X=t) * P(X=t) =0.045$ $P(Y=f | X=f,Z=f) * P(Z=f | X=f) * P(X=f) =0.4005$  The Marginal: $P(Z=t) = 0.5$ and $P(Z=f) = 0.5$ $P(Y=t) = 0.504$ and $P(Y=f) = 0.496$  I'm sorry I cannot present the comment with a clear formating. Thanks  
  > **username_1**: Your joint and marginal distributions are correct. U(Y) is correct as well. For U(Y|X), that is equal to U(X, Y) - U(X), which gives me 0.68867. I think that's where your calculations are off.  
  > **username_0**: Thanks @username_1,  I calculated $U(X,Y)$, but, it seems even off from the result: 1.) Find $P(X,Y)$ by marginalizing $Z$. This gives me $P(X,Y)=0.494$. 2.) Apply $0.494 * log_{2} \frac{1}{0.494}$. This gives me $U(X,Y) = 0.5026$. 3.) Then $U(Y|X)=U(X,Y)-U(X)=(-4973959)$.  Previously, I calculated $U(Y|X)$, as followed: 1.) Calculate $P(X=t) * U(Y|X=t) = 0.5*0.8400385= 0.42002$. 2.) Calculate $P(X=f) * U(Y|X=f) = 0.5*0.8486344= 0.42432$. 3.) Then $U(Y|X)=0.84434$. 4.) Finally, $MI(X,Y) = U(Y) - U(Y|X) = 0.9999538 - 0.84434 = 0.15562$, which half of the correct result.  Thanks so much.  
  
  ---
  
  