Example 2. How to find out how many fish are in the pond?

We catch $n_{a}=20$ fish, mark them, and release them back into the pond. After some time, sufficient for the marked fish to disperse throughout the pond, we catch $m=50$ fish. Suppose that among them, $k_{1}=7$ are marked. Determine the number of fish in the pond $N$ using the method of maximum likelihood.
## Solution.

1. The random variable $\xi$ - the number of marked fish among $m$ caught - is determined by the probabilities

$$
\mathrm{P}(\xi=k)=p(k ; N)=\frac{C_{n_{a}}^{k} C_{N-n_{a}}^{m-k}}{C_{N}^{m}}=\frac{C_{20}^{k} C_{N-20}^{50-k}}{C_{N}^{50}}(k=0,1, \ldots, 50)
$$

2. We define the likelihood function

$$
L(N)=\frac{C_{20}^{7} C_{N-20}^{43}}{C_{N}^{50}}
$$

In this case, the sample consists of a single number $k_{1}=7$.

3. We find the unknown parameter $N$ (the number of fish in the pond) from the condition

$$
L(N) \longrightarrow \max
$$

i.e., the value of $N$ for which the likelihood function $L(N)$ has the maximum value.

The function $L(N)$ is not differentiable, as its argument $N$ takes only discrete values $63,64, \ldots$ Therefore, the value of $N$ for which $L(N)$ reaches its maximum value is found by enumeration.

We have

$$
L(N)=C_{20}^{7} \frac{(N-20)!50!(N-50)!}{43!(N-63)!N!}=A \frac{(N-50) \cdot \ldots \cdot(N-62)}{N(N-1) \cdot \ldots \cdot(N-19)}
$$

where $A=\frac{50!}{43!} C_{20}^{7}$ is an insignificant multiplier. A simple program provides a table of values of $L(N)$ for $N=63 \div 1000$. In the table, we find that $L(N)$ reaches its maximum value at $N=142$.

Answer. $\quad N=142$.

Conditions of the problems. Using the method of maximum likelihood, find the parameters of the distribution based on the sample.

1. Binomial distribution $p(k ; n, p)=C_{n}^{k} p^{k} q^{n-k}, \quad n=10$, sample: $6,5,9,5,8,7,9,6,6,7$.
2. Poisson distribution $p(k ; a)=e^{-a} a^{k} / k!$, sample: $5,3,3,3,5,5,1,2,3,0$.
3. Geometric distribution $p(k ; q)=(1-q) q^{k}$, sample: $1,2,0,4,1,4,4,1,2,2$.
4. Hypergeometric distribution $p\left(k ; n_{A}, n_{B}, m\right)=\frac{C_{n_{a}}^{k} C_{n_{b}}^{m-k}}{C_{n_{a}+n_{b}}^{m}}$, $m=10$, sample: $6,3,7,6,4,4,5,5,6,8$.
5. Distribution of stellar multiplicity $p(k ; q)=(1-q)^{2} k q^{k-1}$, sample: $1,2,1,3,2,1,1,2,1,2$.
6. Uniform distribution $p(x ; a, b)=\frac{1}{b-a}$ for $a \leq x \leq b$, $p(x ; a, b)=0$ for $x < a$ or $x > b$, sample:

$0.35,1.83,0.14,0.64,2.08,-0.46,2.0,-0.31,-0.30,1.87$.

7. Exponential distribution $p(x ; \lambda)=\lambda e^{-\lambda x}$ for $x>0$, $p(x ; \lambda)=0$ for $x \leq 0$, sample:

$1.63,1.35,1.80,1.51,1.56,2.10,1.36,1.48,1.24,1.39$.

8. Rayleigh distribution $p(x ; t)=\frac{x}{t^{2}} e^{-x^{2} / (2 t^{2})}$ for $x>0$, $p(x ; t)=0$ for $x \leq 0$, sample:

$1.63,1.35,1.80,1.51,1.56,2.10,1.36,1.48,1.24,1.39$.

9. Gamma distribution $p(x ; \lambda, \alpha)=\frac{\lambda^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\lambda x}$ for $x>0$, $p(x ; \lambda, \alpha)=0$ for $x \leq 0$, sample:

$1.63,1.35,1.80,1.51,1.56,2.10,1.36,1.48,1.24,1.39$.

10. Weibull distribution $p(x ; a, \alpha)=\frac{\alpha}{a} \left(\frac{x}{a}\right)^{\alpha-1} e^{-(x/a)^{\alpha}}$ for $x \geq a$, $p(x ; a, \alpha)=0$ for $x < a$, sample:

$1.63,1.35,1.80,1.51,1.56,2.10,1.36,1.48,1.24,1.39$.

Answers. 1. $p=0.68$. 2. $a=3.235$. 3. $q=0.677$. 4. $n_{a}=17$, $n_{b}=25$. 5. $q=0.23$. 6. $a=-0.46, b=2.08$. 7. $\lambda=1.17$. 8. $t=1.8$. 9. $\lambda=3.60$. 10. $a=1.24, \alpha=2.88$.

### 7.13. Method of Least Squares

PROBLEM STATEMENT. To determine the parameters $a_{1}, \ldots, a_{j}$ in the formula $y=f\left(x ; a_{1}, \ldots, a_{j}\right)$, values of $y$ were measured at different values of $x$. The sample obtained is $\left(x_{1}, y_{1}\right), \ldots,\left(x_{n}, y_{n}\right)$. Using these data, determine the parameters $a_{1}, \ldots, a_{j}$ by the method of least squares. Errors in the measurements of $x_{1}, \ldots, x_{n}$ and computational errors can be neglected.

PLAN OF SOLUTION. The quantities $\xi_{k}=y_{k}-f\left(x_{k} ; a_{1}, \ldots, a_{j}\right)$ are non-zero due to measurement errors in $y_{k}$, since errors in the measurements of $x_{k}$ and the computation of $f\left(x_{k} ; a_{1}, \ldots, a_{j}\right)$ can be neglected. It is assumed that measurement errors are normally distributed with a mean of 0 (no systematic errors) and some variance $\sigma^{2}$, determined by the accuracy of the instrument. Therefore, the likelihood function has the form

$$
\begin{aligned}
L\left(a_{1}, \ldots, a_{j}\right)= & \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\xi_{1}^{2} /\left(2 \sigma^{2}\right)} \cdot \ldots \cdot \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\xi_{n}^{2} /\left(2 \sigma^{2}\right)}= \\
& =\frac{1}{\left(2 \pi \sigma^{2}\right)^{n / 2}} \exp \left[-\frac{1}{2 \sigma^{2}} \sum_{k=1}^{n}\left(y_{k}-f\left(x_{k} ; a_{1}, \ldots, a_{j}\right)\right)^{2}\right]
\end{aligned}
$$

This function reaches its maximum value for those $a_{1}, \ldots, a_{j}$ for which

$$
S\left(a_{1}, \ldots, a_{j}\right)=-\ln L\left(a_{1}, \ldots, a_{j}\right)=\sum_{k=1}^{n}\left(y_{k}-f\left(x_{k} ; a_{1}, \ldots, a_{j}\right)\right)^{2}
$$

reaches its minimum value.

1. We find the partial derivatives

$$
\frac{\partial S}{\partial a_{k}} \quad(k=1, \ldots, j)
$$

2. We determine the parameters $a_{1}, \ldots, a_{j}$ as the solution to the system of equations

$$
\frac{\partial S}{\partial a_{1}}=0, \quad \frac{\partial S}{\partial a_{2}}=0, \ldots, \frac{\partial S}{\partial a_{j}}=0
$$

Remarks.

1. After the parameters $a_{1}, \ldots, a_{j}$ are determined,