Example 1.10.5. Let \(a, b, c\) be non-negative real numbers such that \(a+b+c=3\). For each \(k \geq 0\), find the maximum value of
\[a^{2}(k b+c)+b^{2}(k c+a)+c^{2}(k a+b)\]
SOLUTION. Because the expression is cyclic, we can assume first that \( c=0, a+b=3 \) and find the maximum value of
\[ F=k a^{2} b+b^{2} a \]

For \( k=1 \), we have \( F=a b(a+b)=3 a b \leq \frac{27}{4} \) by AM-GM inequality. Otherwise, assume that \( k \neq 1 \). We denote
\[ f(a)=k a^{2}(3-a)+a(3-a)^{2}=(1-k) a^{3}+3(k-2) a^{2}+9 a . \]

We have
\[ f^{\prime}(a)=3(1-k) a^{2}+6(k-2) a+9 \]

Now we check the roots of the derivative and use that \( a \in[0,3] \)
\[ f^{\prime}(a)=0 \Leftrightarrow a=a_{0}=\frac{2-k-\sqrt{k^{2}-k+1}}{1-k}=\frac{\sqrt{k^{2}-k+1}+k-2}{k-1} . \]

The maximum of \( f(a) \) is attained for \( a=a_{0} \) and
\[ \max _{a \in[0,3]} f(a)=f\left(a_{0}\right)=\frac{2\left(k^{2}-k+1\right)}{(k-1)^{2}}\left(\sqrt{k^{2}-k+1}+k-2\right)+\frac{3(k-2)}{k-1} . \]

According to CD3 theorem, we conclude that the maximum of
\[ a^{2}(k b+c)+b^{2}(k c+a)+c^{2}(k a+b) \]
is
\[ \min \left\{3(k+1) ; \frac{2\left(k^{2}-k+1\right)}{(k-1)^{2}}\left(\sqrt{k^{2}-k+1}+k-2\right)+\frac{3(k-2)}{k-1}\right\} \]