Find the equations of the circles tangent to the lines A and B, and their centers are on the line C. I can't find the solution to this problem, whatever the angle I look at it.
We need to find the equations of the circles tangent to two lines (A and B), knowing that their centers are on the line C.
$$
A = x - 2y + 2 = 0
$$
$$
B = 2x-y+1=0
$$
$$
C = 5x-y-11=0
$$

The key fact here is that the radius of a circle is perpendicular to the corresponding tangent line.
If we want the circles tangent to both lines $A$ and $B$, with centres on line $C$ then we want points $P$ on the line $C$ whose perpendicular distance from line $A$ and line $B$ is equal. 
The perpendicular distance of a point $(p,q)$ from the line $ax+by+c=0$ is given by
$$\frac{|ap+bq+c|}{\sqrt{a^2+b^2}}$$
The line $C$ has equation $5x-y-11=0$, i.e. $y=5x-11$ and so a general point is parametrised by $(p,q)=(t,5t-11)$, for some parameter $t$.
Line $A$ has $a=1$, $b=-2$ and $c=2$. Line $B$ has $a=2$, $b=-1$ and $c=1$.
It follows that we need to solve, for $t$, the following equation:
\begin{eqnarray*}
\frac{|1t+(-2)(5t-11)+2|}{\sqrt{1^2+(-2)^2}} &=& \frac{|2t+(-1)(5t-11)+1|}{\sqrt{2^2+(-1)^2}} \\ \\
\frac{|24-9t|}{\sqrt 5} &=& \frac{|12-3t|}{\sqrt 5}
\end{eqnarray*}
Solving $|24-9t|=|12-3t|$ gives $t=2$ or $t=3$, and the centre of the circles are $(p,q)=(2,-1)$ and $(p,q)=(3,4)$.
The radius is given by substituting $t=2$ and $t=3$ into either of the perpendicular distance formulae:
\begin{eqnarray*}
\frac{|24-9(2)|}{\sqrt 5} &=& \frac{6}{\sqrt 5} \\ \\
\frac{|24-9(3)|}{\sqrt 5} &=& \frac{3}{\sqrt 5}
\end{eqnarray*}
The equations of the circles are given by $(x-2)^2+(y+1)^2=\frac{36}{5}$ and $(x-3)^2+(y-4)^2=\frac{9}{5}$.
Here's a picture. The lines $A$ and $B$ are in black. The line $C$ is in red. The circles, and their centres are in blue.

