Simplify
\[\frac{\sec x}{\sin x} - \frac{\sin x}{\cos x}.\]
We can write
\begin{align*}
\frac{\sec x}{\sin x} - \frac{\sin x}{\cos x} &= \frac{1}{\cos x \sin x} - \frac{\sin x}{\cos x} \\
&= \frac{1 - \sin^2 x}{\cos x \sin x} \\
&= \frac{\cos^2 x}{\cos x \sin x} \\
&= \frac{\cos x}{\sin x} \\
&= \boxed{\cot x}.
\end{align*}