Question: For example, several missions have been launched just to find ice at the lunar poles. Why do astronomers and space agencies care so much about there being ice somewhere?

Answer: 
It will cost a lot to send even a few liters of water to the Moon, hundreds of thousands of dollars per liter. Water is also probably the best source of breathable air. Humans on the Moon (or on Mars) will need to consume quite a bit of water per day, so any water obtained *in situ* on the Moon (or on Mars) will represent a significant savings.
Water is the big one.

Why?

Because it's made up of Hydrogen and Oxygen, which in turn are the primary materials needed for ongoing spaceflight.

Hydrogen/Oxygen can be liquefied and used as rocket fuel.

We need oxygen to breathe, water to drink, and you can use both to grow more food.

So as long as you have water and a way to generate power, you have all the raw materials you need to operate a space base. Or a refueling station. Or whatever you want.