

# Basic concepts of probability: sample space, event, and probability measure

Before diving into the concepts of expectation and variance, let's first understand the basic concepts of probability. Probability theory is a branch of mathematics that deals with the analysis of random events and the study of the likelihood of different outcomes.

A **sample space** is a set of all possible outcomes of a random experiment. It is denoted by $\Omega$. For example, if we are rolling a fair six-sided die, the sample space would be $\Omega = \{1, 2, 3, 4, 5, 6\}$.

An **event** is a subset of the sample space. It is a set of outcomes that have some specific property or satisfy some condition. For example, if we want to find the event of rolling an even number, it would be $E = \{2, 4, 6\}$.

A **probability measure** is a function that assigns a probability to each event in the sample space. It satisfies certain properties, such as non-negativity and normalization. The probability of an event $E$ is denoted by $P(E)$. For example, if we want to find the probability of rolling an even number, it would be $P(E)$.

## Exercise
Calculate the probability of rolling an even number on a fair six-sided die.

# Bayes theorem and its applications in probability

Bayes theorem is a fundamental result in probability theory that relates the probabilities of two events based on their conditional probabilities. It is named after the Reverend Thomas Bayes, who published it in 1763.

The theorem states that for any two events $A$ and $B$, with $P(B) \neq 0$, we have:

$$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$$

This theorem is useful in updating our beliefs about the probability of an event based on new information or evidence.

## Exercise
Suppose you have a bag with 3 red balls and 2 blue balls. You draw a ball from the bag and it turns out to be red. What is the probability that the next ball you draw is also red, given that you know the first ball was red?

# Conditional probability and its properties

Conditional probability is the probability of an event given that another event has occurred. It is denoted by $P(A \mid B)$.

The conditional probability of an event $A$ given another event $B$ is defined as:

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$

There are several properties of conditional probability that are useful in solving problems:

- $P(A \mid B) = P(B \mid A)$
- $P(A \mid B) = \frac{P(A \cap B)}{P(B)} = \frac{P(B \cap A)}{P(B)} = P(B \mid A)$
- $P(A \cap B) = P(A \mid B) P(B)$
- $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

## Exercise
Calculate the conditional probability of rolling an even number on a fair six-sided die, given that the first roll was red.

# Expectation: definition and properties

Expectation, or the mean, is a measure of the central tendency of a probability distribution. It is the average value that a random variable is expected to take.

The expectation of a discrete random variable $X$ with probability mass function $P(X = x)$ is defined as:

$$E(X) = \sum_{x} x P(X = x)$$

The expectation of a continuous random variable $X$ with probability density function $f(x)$ is defined as:

$$E(X) = \int_{-\infty}^{\infty} x f(x) dx$$

Some properties of expectation are:

- Linearity: $E(aX + b) = aE(X) + b$
- Expected value is additive for independent random variables: $E(X + Y) = E(X) + E(Y)$
- Expected value is additive for independent events: $E(X \mid A) = E(X) + P(A \mid X)$

## Exercise
Calculate the expectation of rolling a fair six-sided die.

# Variance: definition and properties

Variance is a measure of the dispersion or spread of a probability distribution. It quantifies the extent to which individual data points in a set deviate from the mean.

The variance of a discrete random variable $X$ with probability mass function $P(X = x)$ is defined as:

$$Var(X) = E((X - E(X))^2)$$

The variance of a continuous random variable $X$ with probability density function $f(x)$ is defined as:

$$Var(X) = E((X - E(X))^2) = \int_{-\infty}^{\infty} (x - E(X))^2 f(x) dx$$

Some properties of variance are:

- Non-negativity: $Var(X) \geq 0$
- $Var(aX + b) = a^2 Var(X)$
- $Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y)$

## Exercise
Calculate the variance of rolling a fair six-sided die.

# Independence and its properties in probability

Independence is a property of two events that states whether the occurrence of one event affects the probability of the other event.

Two events $A$ and $B$ are independent if:

$$P(A \cap B) = P(A) P(B)$$

Some properties of independence are:

- $P(A \mid B) = P(A)$ if $A$ and $B$ are independent
- $P(A \cap B) = P(A) P(B)$
- $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

## Exercise
Determine if the events "rolling an even number" and "rolling a number greater than 3" are independent.

# Probability distributions: discrete and continuous

Probability distributions are mathematical functions that describe the possible outcomes of a random experiment and their corresponding probabilities.

- **Discrete probability distributions**: These distributions have a countable number of outcomes and are defined over a discrete set. Examples include the binomial distribution, Poisson distribution, and geometric distribution.

- **Continuous probability distributions**: These distributions have an uncountable number of outcomes and are defined over a continuous set. Examples include the normal distribution, exponential distribution, and uniform distribution.

## Exercise
Find the probability distribution of rolling a fair six-sided die.

# Applications of expectation, variance, and independence in probability distributions

Expectation, variance, and independence have numerous applications in probability distributions.

- Expectation is used to find the average value of a random variable, which is useful in understanding the central tendency of a distribution.
- Variance is used to measure the dispersion of a distribution, which is useful in understanding the spread of a distribution.
- Independence is used to simplify calculations and make predictions based on the occurrence of one event without considering the occurrence of another event.

## Exercise
Find the expectation, variance, and check for independence of rolling a fair six-sided die.

# Limit theorems and their applications in probability theory

Limit theorems are mathematical results that describe the behavior of the sum or average of a large number of independent and identically distributed random variables.

- **Weak Law of Large Numbers**: If a sequence of random variables $X_1, X_2, \ldots$ are independent and identically distributed with finite mean $\mu$, then the sample mean $S_n = \frac{1}{n} \sum_{i=1}^n X_i$ converges in probability to $\mu$ as $n \to \infty$.

- **Central Limit Theorem**: If a sequence of random variables $X_1, X_2, \ldots$ are independent and identically distributed with finite mean $\mu$ and variance $\sigma^2$, then the sample mean $S_n = \frac{1}{n} \sum_{i=1}^n X_i$ converges in distribution to a normal distribution with mean $\mu$ and variance $\frac{\sigma^2}{n}$ as $n \to \infty$.

## Exercise
Apply the Weak Law of Large Numbers to the sum of rolling a fair six-sided die.

# Central limit theorem and its applications

The Central Limit Theorem states that the sum of a large number of independent and identically distributed random variables, each with a finite mean and variance, converges in distribution to a normal distribution.

This theorem has numerous applications in probability theory and real-world problems. For example, it can be used to analyze the distribution of the sum of random variables, predict the behavior of a large sample, and understand the spread of data.

## Exercise
Apply the Central Limit Theorem to the sum of rolling a fair six-sided die.

# Real-world examples and case studies

Probability theory has numerous applications in real-world problems. Here are some examples:

- **Gambling**: Probability theory is used to analyze the likelihood of winning or losing in games of chance, such as card games, lotteries, and sports betting.

- **Insurance**: Probability theory is used to determine the risk associated with various events and to set insurance premiums.

- **Statistics**: Probability theory is used to analyze and interpret data, make predictions, and test hypotheses.

- **Engineering**: Probability theory is used to design and analyze systems with random components, such as communication networks, queuing systems, and manufacturing processes.

- **Finance**: Probability theory is used to model financial markets, evaluate investment strategies, and predict stock prices.

## Exercise
Apply probability theory to analyze the risk of a lottery ticket.