How to do this without using trigonometric substitution? $$\int \frac {\rho^2}{(\rho^2+ h^2)^\frac 32} d\rho$$

In addition to the pythagorean identities for circular trig functions, such as
$$ 1 + \tan^2 x = \sec^2 x $$
and for the hyperbolic trig functions, such as
$$ 1 + \sinh^2 x = \cosh^2 x $$
there is a similar identity involving rational functions:
$$ 1 + \left( \frac{y^2 - 1}{2y} \right)^2 = \left( \frac{y^2 + 1}{2y} \right)^2 $$
The various forms of this identity can be computed as needed from
$$ (2y)^2 + (y^2 - 1)^2 = (y^2 + 1)^2 $$
So, for example, when one has
$$ \sqrt{1 + x^2} \, \mathrm{d} x$$
one could make the substitution
$$x = \frac{y^2 - 1}{2y} \qquad \qquad y > 0$$
to get
$$ \left( \frac{y^2 + 1}{2y} \right) \cdot \left( \frac{1}{2} \left( 1 + \frac{1}{y^2} \right) \mathrm{d} y \right) $$
With the square root eliminated, the problem can then be solved via partial fractions. Although in this example the denominator is a power of $y$, so you have the much simpler method of just expanding everything.
Similarly, one could substitute $x = \frac{2y}{y^2 - 1}$ instead.
