How can a 'tall-and-skinny' property related to kurtosis be stated rigorously in terms of its (smooth) probability functions? Skewness is generally defined as a standardized third-order centered moment.
$$S(X) \triangleq \frac{\mathbb{E}\left[ \left( X - \mathbb{E} \left[  X \right] \right)^3 \right]}{\left( \mathbb{E}\left[ \left( X - \mathbb{E} \left[ X \right] \right)^2 \right] \right)^{\frac{3}{2}}}$$
Sometimes skewness is the property that we're interested in, but often it is used to quantify the reflective asymmetry of a probability distribution. This is a heuristic because Meijer 2000 demonstrated that asymmetric distributions can have a skewness of zero even though a non-zero skewness implies that a distribution is asymmetric.
Drawing from skewness as an analogy, I would like to consider kurtosis under a similar light. Kurtosis is likewise a high-order moment:
$$K(X) \triangleq \frac{\mathbb{E}\left[ \left( X - \mathbb{E} \left[  X \right] \right)^4 \right]}{\left( \mathbb{E}\left[ \left( X - \mathbb{E} \left[ X \right] \right)^2 \right] \right)^{2}}$$
Analogously, as skewness tells us about reflective symmetry, kurtosis tells us about how 'tall-and-skinny' the distribution is. As whuber discusses on another post, the symmetry property can be formulated as an equality in terms of either its CDF or PDF (assuming sufficient smoothness).
I have yet to see a formulation of the 'tall-and-skinny' property of a distribution written in an analogous way in terms of either the CDF or PDF of a distribution.
Off the top of my head I would speculate that kurtosis of a CDF could involve a comparison to either extreme of a step function being a maximally-kurtic peak, and a line being a minimally-kurtic peak.
In terms of the PDF I might suspect that an aspect-ratio could be suitable, except that some choice on what interval counts as the 'base' would need to be defined. The choice of height might be taken to be the density at the maximum likelihood value (MLV), but this would be problematic for distributions with multiple MLV's.
How can a 'tall-and-skinny' property related to kurtosis be stated rigorously in terms of its (smooth) probability functions?

Well, you cannot relate tall and skinny to kurtosis because there is no mathematical connection. The beta(.5,1) distribution is infinitely tall and skinny but has low kurtosis. And the .9999U(0,1) + .0001Cauchy mixture appears perfectly flat over 99.99% of the observable data, but has infinite kurtosis.
Contrary to whuber's comment, kurtosis is in fact precisely related to the tails. Larger kurtosis mathematically implies greater tail leverage. See here and here for precise descriptions of "tail leverage."
