Question: This may sound like a rudimentary question, but I am curious if dummy variables reduce or eliminate the need for certain other controls. For instance, if I am looking at the impact of some variable X\_1 that is at a CITY level (across several countries) and want to use some X\_2 as a control - if X\_2 is only available at a COUNTRY level, then wouldn't a COUNTRY dummy variable capture all relevant variance anyway and eliminate the need for X\_2 as a control in estimating X\_1?

Answer: 
No it is not guaranteed it will solve the problem. The fixed country dummy would capture all relevant effects that are time invariant.

Now in principle you could also add time fixed effects but that’s still no panacea because time fixed effects assume the time effect is spatially homogenous.

If you believe that $X\_2$ is something that varies across time and has heterogenous effect (e.g. output of firm, number of pupils per school etc) then you still need to control for $X\_2$.
My answer is "Yes" for your specific question with $X\_2$ being available only at the country level. (No if $X\_2$ is a general variable.)

Country dummies eliminate the need for $X\_2$ as a control because $X\_2$ is perfectly predicted by the country dummies. More technically, $X\_2$ and $m-1$ country dummies ($m$ = number of countries) are perfectly collinear. If you regress $y$ on $X\_1$, $X\_2$ and country dummies, $X\_2$ or one country dummy will be omitted depending on which of `x2` and `i.country` is specified first.

That said, please make sure that the two models (one with $X\_2$ and without country dummies, and the other with country dummies) are *two different models* because one controls for only $X\_2$ while the other controls for all cross-country heterogeneity. You can also compare the $X\_1$ coefficients from both models for whether $X\_2$ is sufficient as a control, which can be interesting.