5. Let $\mathcal{C}$ be a circle in the $x y$ plane with radius 1 and center $(0,0,0)$, and let $P$ be a point in space with coordinates $(3,4,8)$. Find the largest possible radius of a sphere that is contained entirely in the slanted cone with base $\mathcal{C}$ and vertex $P$.
Answer: $3-\sqrt{5}$ Consider the plane passing through $P$ that is perpendicular to the plane of the circle. The intersection of the plane with the cone and sphere is a cross section consisting of a circle inscribed in a triangle with a vertex $P$. By symmetry, this circle is a great circle of the sphere, and hence has the same radius. The other two vertices of the triangle are the points of intersection between the plane and the unit circle, so the other two vertices are $\left(\frac{3}{5}, \frac{4}{5}, 0\right),\left(-\frac{3}{5},-\frac{4}{5}, 0\right)$.
Using the formula $A=r s$ and using the distance formula to find the side lengths, we find that $r=\frac{2 A}{2 s}=\frac{2 * 8}{2+10+4 \sqrt{5}}=3-\sqrt{5}$.