text: ```Administrative info
  PA2 due Monday
  HW5 due Tuesday
  
  Review
  Recall that a probability space consists of the following:
  (1) A random experiment.
  (2) The sample space (set of possible outcomes).
  (3) The probability of each possible outcome of the experiment.
  Further recall that the probabilities must satisfy the following:
  (1) ∀ω∈Ω . 0 <= Pr[ω] <= 1
  (2) ∑_{ω∈Ω} Pr[ω] = 1
  An event is a subset of the sample space, i.e. a set of outcomes
  from the sample space. The probability of an event E is the sum of
  the probabilities of the outcomes in E:
  Pr[E] = ∑_{ω∈E} Pr[ω].
  In the case of a uniform distribution, this simplifies to
  Pr[E] = |E|/|Ω|.
  
  Probability Identities
  Before we move on, let's note some facts about probability that can
  make it easier to compute probabilities.
  
  We defined the complement of an event E as E = &Omega\E. Then,
  Pr[E] = 1 - Pr[E]. Proof:
  ∑_{ω∈Ω} Pr[ω]
  = ∑_{ω∈E;} Pr[ω] +
  ∑_{ω∈Ω\E} Pr[ω]
  = Pr[E] + Pr[E] = 1.
  
  Let A and B be events in Ω. Then Pr[A ∪ B] = Pr[A] + Pr[B]
  - Pr[A ∩ B]. Writing this out in terms of sums, we get
  ∑_{ω∈Ω} Pr[ω]
  = ∑_{ω∈A;} Pr[ω] +
  ∑_{ω∈B} Pr[ω] -
  ∑_{ω∈A ∩ B} Pr[ω]
  As in inclusion/exclusion for sets, the first two terms double count
  the probabilities of those outcomes in A ∩ B, so we have to
  subtract the probability of A ∩ B.
  
  EX: What is the probability that a random integer n between 1 and
  100 is divisible by 5 or 7?
  ANS: Let A be the event that n is divisible by 5, B be the event
  that it is divisible by 7. Pr[A] = 1/5, Pr[B] = 14/100 = 7/50,
  and Pr[A ∩ B] = 2/100 = 1/50. So Pr[A ∪ B] = 1/5 + 7/50
  - 1/50 = 16/50 = 8/25.
  
  Let A_1, ..., A_n be n mutually disjoint events in Ω. Then
  Pr[A_1 ∪ ... ∪ A_n] = Pr[A_1] + ... + Pr[A_n]. This follows
  from the above, generalized to n events using induction, and then
  removing the intersection terms which are all 0.
  
  EX: Suppose I roll a red and a blue die. What is the probability that
  the red die is less than 4?
  ANS: Let A_i be the event that the red die is i. Then Pr[A_i] = 1/6
  for 1 <= i <= 6, and the A_i are mutually disjoint. Thus,
  Pr[A_1 ∪ A_2 ∪ A_3] = Pr[A_1] + Pr[A_2] + Pr[A_3] =
  1/2.
  
  Conditional Probability
  A pharmaceutical company is marketing a new test for HIV that it
  claims is 99% effective, meaning that it will report positive for
  99% of people who have HIV and negative for 99% of those who don't
  have HIV. Suppose a random person takes the test and gets a positive
  test result. What is the probability that the person has HIV?
  
  This is an example of conditional probability. Given some
  information about a particular event in the sample space, we want to
  compute new probabilities for other events.
  
  Let's start off with simpler examples before coming back to the
  above.
  
  EX: Suppose I flip a fair coin twice. The result of the first flip
  is heads. What is the probability that I got two heads?
  ANS: Let's start by drawing the sample space Ω. There are 4
  equally likely outcomes HH, HT, TH, and TT. We are now told
  that event A = "the first flip is H" has occurred. Which
  outcomes are now possible? There are only 2 outcomes in A, HH
  and HT, each of which is equally likely. So we have a new
  sample space Ω' that consists of just the outcomes HH and
  HT, each with probability 1/2. Let event B = "both flips are
  heads." What is the probability of B in this new sample space?
  Only one of the two outcomes in Ω' is in B, so Pr[B] =
  1/2 in the new sample space. We write this as Pr[B|A], "the
  probability of B given A," which is the probability of B
  occurring in a new sample space consisting of just those
  outcomes in A.
  
  Generalizing the above procedure, suppose we are told an event A
  occurs. Then what is the new conditional probability of each outcome
  ω, i.e. Pr[ω|A]? For ω ∉ A, this is clearly
  0. For ω ∈ A, the relative likelyhood of any two outcomes
  in A should remain the same, but we need to renormalize so that we
  satisfy the requirement that all probabilities add to 1. By
  definition, we had ∑_{ω ∈ A} Pr[ω] = Pr[A], so
  if we normalize by dividing by Pr[A], i.e. Pr[ω|A] =
  Pr[ω]/Pr[A], we get ∑_{ω ∈ A} Pr[ω|A] =
  ∑_{ω ∈ A} Pr[ω]/Pr[A] = Pr[A]/Pr[A] = 1.
  
  Now suppose we have another event B. What is Pr[B|A]? The outcomes
  in B that are not in A contribute nothing, since their new
  conditional probabilities are 0. So only the outcomes in both B and
  A contribute any probability, and we get Pr[B|A] = ∑_{ω
  ∈ B ∩ A} Pr[ω|A] = ∑_{ω ∈ B ∩ A}
  Pr[ω]/Pr[A] = Pr[B ∩ A]/Pr[A].
  
  To summarize, when conditioning on an event A, we cross out any
  possibilities that are incompatible with A and then renormalize by
  1/Pr[A] so that the probabilities of the remaining outcomes add to
  1. We can compute the probabilities of events directly in this new
  sample space or use the identities above to get the same result.
  
  EX: Suppose I toss a red and a blue die, and I tell you that the
  resulting sum is 4. What is the probability that the red die is
  1?
  ANS: Let A be the event that the sum is 7, B be the event that the
  red die is 1. The outcomes (1, 3), (2, 2), and (3, 1) are in A,
  so Pr[A] = 1/12. What is Pr[B ∩ A]? Only the outcome (1, 3)
  is in B ∩ A, so Pr[B ∩ A] = 1/36. Then Pr[B|A] = Pr[B
  ∩ A]/Pr[A] = 1/3.
  We could also have redefined the sample space to come up with
  the same result. Given A, we have a new sample space Ω'
  consisting of the outcomes (1, 3), (2, 2), and (3, 1), each
  with probability 1/3. Then B has probability 1/3 in this new
  sample space. So Pr[B|A] = 1/3.
  
  EX: Suppose I toss a red and a blue die, and I tell you that the
  resulting sum is 7. What is the probability that the red die is
  1?
  ANS: Let A be the event that the sum is 7, B be the event that the
  red die is 1. Pr[A] = 1/6 as we computed before. What is Pr[B
  ∩ A]? Only the outcome (1, 6) is in B ∩ A, so Pr[B
  ∩ A] = 1/36. Then Pr[B|A] = Pr[B ∩ A]/Pr[A] = 1/6.
  
  EX: Suppose I toss 3 balls into 3 bins (with replacement). Let
  A = "1st bin empty," B = "2nd bin empty." What is Pr[A|B]?
  ANS: Pr[B] = 2^3/3^3 = 8/27, Pr[A ∩ B] = 1/3^3 = 1/27, so
  Pr[A|B] = (1/27)/(8/27) = 1/8.
  Thus, the fact that the 2nd bin is empty makes it much less
  likely that the 1st one is as well.
  
  EX: Suppose I flip a fair coin 51 times. If the first 50 flips are
  heads, what is the probability that the 51st is heads?
  ANS: Let A be the event that the first 50 flips are heads, B be the
  event that the 51st is heads. There are only 2 outcomes in A
  out of 2^51, so Pr[A] = 1/2^50. There are 2^50 outcomes in B,
  so Pr[B] = 1/2. Only one outcome is in both A and B, so Pr[A
  ∩ B] = 1/2^51. Then Pr[B|A] = (1/2^51)/(1/2^50) = 1/2.
  So the first 50 flips tell us nothing about the 51st; the
  probability of heads is still 1/2.
  
  We have seen multiple examples where Pr[B|A] = Pr[B]. We say that A
  and B are "independent" of this is the case. Intuitively, two events
  A and B are independent of knowing that one happens does not change
  the likelihood of the other happening. So the 51st flip of a fair
  coin is independent from what came up before.
  
  If A and B are independent, we get Pr[B|A] = Pr[B ∩ A]/Pr[A] =
  Pr[B], so Pr[B ∩ A] = Pr[A] Pr[B]. This is a very useful
  identity.
  
  EX: Suppose I flip a coin with probability p of heads n times. What
  is the probability of a particular outcome with k heads?
  ANS: Each flip is independent, with probability p for heads. The k
  heads flips have probability p, and the n-k tails flips have
  probability (1-p). So we get p^k (1-p)^(n-k) for an outcome
  with k heads.
  
  EX: Suppose a casino advertises the following game. You pick a
  number from 1 to 6. The casino rolls three dice, and if your
  number comes up, you win. What is your probability of winning?
  ANS: It's not 1/2! Let A_i be the event that your number comes up
  on the ith die. We want to know
  Pr[A_1 ∪ A_2 ∪ A_3]
  = 1 - Pr[A_1 ∩ A_2 ∩ A_3]
  = 1 - Pr[A_1] Pr[A_2] Pr[A_3]
  = 1 - (5/6)^3 ≈ 1 - 0.58 = 0.42.
  In the third line above, we used the fact that the results of
  each dice are mutually independent. We will come back to the
  concept of mutual independence later.
  So your probability of winning is less than 1/2.
  
  Suppose you are flying to Las Vegas (in order to play the game
  above). Your friend, fearing for your safety, gives you the
  following advice: "You know, you should always carry a bomb on an
  airplane. The chance of there being one bomb on the plane is pretty
  small, but the chance of two bombs is miniscule. So by carrying a
  bomb on the airplane, your chances of being blown up are
  astronomically reduced." What do you think of his advice?
  
  Let A be the event that you carry a bomb on board, B be the event
  that someone else carries a bomb on board. How are A and B related?
  They are independent, so Pr[B|A] = Pr[B], and the likelihood that
  someone else has a bomb doesn't change one bit if you bring one
  aboard.
  
  ```
metadata: {"found_math": false, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0}