Generative Models - Class Conditional Density and Posterior Probability In the section 1.5.4 of Bishop's PRML book, a brief description of generative models is given. For classification decisions, it is stated that "the class-
conditional densities may contain a lot of structure that has little effect on the posterior probabilities". Here is the figure used to demonstrate this -


*

*What does it mean by "a lot of structure" in this context? Is it equivalent to saying that the class-conditional densities have much more information than required to determine the posterior probabilities?

*By looking at the figure, how can we conclude that "a lot of structure" of the class-conditional probabilities have little effect on the posterior probabilities?


Have a look at the class-conditional densities in the left plot. There is some information to be taken from it:

*

*$p(x|C_1)$ is bimodal, with a peak at x=0.2 and one at x=0.5, separated by a "valley" at roughly x=0.3

*$p(x|C_2)$ is unimodal and symmetric, with it's center at roughly x=0.7

*$p(x|C_1)$ and $p(x|C_2)$ intersect near x=0.58

*$p(x|C_1)$ is approaching zero on the left hand side at around x=0.8, $p(x|C_2)$ is doing so on the left hand side near x=0.3

The curve of the posterior probability that we see on the right side is the same for both classes, but mirrored. On the left the two classes clearly differ in their modality (point 1. and 2.), but this structural information had no impact on the shape of the posterior.
Only the last two points are reflected in the probabilities that we see on the right:

*

*Since the conditional densities intersect the way they do at aroun 0.58, the conditional probability has to be 0.5 at the intersection.

*Because the conditional densities approach zero in the areas mentioned above, the posterior probability of the other class is approaching 1 around that point.

