Problem 4. Point $O$ lies inside an isosceles right triangle $A B C$. The distance from it to vertex $A$ of the right angle is 6, to vertex $B$ is 4, and to vertex $C$ is 8. Find the area of triangle $A B C$.
Answer: $20+6 \sqrt{7}$.

First solution. Consider a $90^{\circ}$ rotation around point $A$ that maps point $C$ to point $B$. Let point $O$ be mapped to point $D$ during this rotation; then segment $BD$ is the image of segment $CO$; since the length of segments does not change during rotation, $BD=CO=8$. We obtain quadrilateral $OADB$, in which $OA=AD=6, BD=8, OB=4$, $\angle OAD=90^{\circ}$ (see diagram). We can reason in several ways from here.

![](https://cdn.mathpix.com/cropped/2024_05_06_c901cfc70ca56cbce9b9g-02.jpg?height=348&width=443&top_left_y=203&top_left_x=1543)

First method. Consider a coordinate system where point $O$ has coordinates $(0,0)$, point $A$ has coordinates $(6,0)$, and point $D$ has coordinates $(6,-6)$. We will find the coordinates of point $B(x, y)$, considering that $OB=4$ and $DB=8$, i.e.,

$$
\left\{\begin{array}{l}
x^{2}+y^{2}=16 \\
(x-6)^{2}+(y+6)^{2}=64
\end{array}\right.
$$

Subtracting the second equation from the first, we get $12x-12y-72=-48$, hence $x-y=2$. Substituting $x=y+2$ into the first equation, we get $2y^{2}+4y-12=0, y^{2}+2y-6=0$, from which $y=-1 \pm \sqrt{7}$. Since (see diagram) point $B$ must lie on the same side of line $OA$ as point $D$, $y>0$, so only the root $y=-1-\sqrt{7}$ is valid, from which $x=1-\sqrt{7}$.

Finally, $S=\frac{AB^{2}}{2}=\frac{(x-6)^{2}+y^{2}}{2}=\frac{(1-\sqrt{7}-6)^{2}+(1+\sqrt{7})^{2}}{2}=20+6 \sqrt{7}$.

Second method. First, note that $OD=6 \sqrt{2}, \angle ODA=45^{\circ}$. Let $\angle ODB=\varphi$. Then, by the cosine rule for triangle $ODB$, we have

$$
\cos \varphi=\frac{OD^{2}+BD^{2}-OB^{2}}{2 \cdot OD \cdot BD}=\frac{(6 \sqrt{2})^{2}+8^{2}-4^{2}}{2 \cdot 6 \sqrt{2} \cdot 8}=\frac{120}{2 \cdot 6 \sqrt{2} \cdot 8}=\frac{5 \sqrt{2}}{8}
$$

and then $\sin \varphi=\sqrt{1-\frac{50}{64}}=\frac{\sqrt{14}}{8}$. Now, by the cosine rule for triangle $ADB$, we get

$$
S=\frac{AB^{2}}{2}=\frac{AD^{2}+BD^{2}-2 \cdot AD \cdot BD \cdot \cos \left(\varphi+45^{\circ}\right)}{2}=18+32-6 \cdot 8 \cdot \frac{\cos \varphi-\sin \varphi}{\sqrt{2}}=20+6 \sqrt{7}
$$

Second solution. Let $AB=AC=x, \angle OAB=\varphi$. By the cosine rule for triangles $OAB$ and $OAC$, we have:

$$
\begin{aligned}
& 4^{2}=x^{2}+6^{2}-12 x \cos \varphi \\
& 8^{2}=x^{2}+6^{2}-12 x \sin \varphi
\end{aligned}
$$

from which

$$
\begin{aligned}
12 x \cos \varphi & =x^{2}+20 \\
12 x \sin \varphi & =x^{2}-28
\end{aligned}
$$

Squaring these inequalities and adding them, we get a quadratic equation in $x^{2}: 144 x^{2}=2 x^{4}-16 x^{2}+1184, x^{4}-80 x^{2}+592=0$. The roots of this equation are

$$
x_{1,2}^{2}=40 \pm 12 \sqrt{7}
$$

Note that $40-12 \sqrt{7}<36$, and in this case $x<AO$, meaning point $O$ will not lie inside the triangle, so $S=\frac{x^{2}}{2}=20+6 \sqrt{7}$.

Plan for the third solution. Reflect point $O$ symmetrically with respect to the sides $AB, AC$, and $BC$ of triangle $ABC$; denote the images by $X, Y$, and $Z$ respectively. (see diagram) Then $AY=AX=AO, CY=CZ=CO, BX=BZ=BO$. By a simple calculation of angles, we see that $\angle YCZ=2 \angle ACB=90^{\circ}, \angle XBZ=2 \angle ABC=90^{\circ}, \angle XAY=2 \angle BAC=180^{\circ}$.

The area of pentagon $XYCZB$ is twice the area of triangle $ABC$. On the other hand, the area of $XYCZB$ consists of the areas of two right triangles $YCZ$ and $XBZ$, in which we know one leg, as well as triangle $XYZ$, for which we know all sides, so we can find its area, for example, using Heron's formula.

![](https://cdn.mathpix.com/cropped/2024_05_06_c901cfc70ca56cbce9b9g-03.jpg?height=742&width=723&top_left_y=186&top_left_x=1246)