Question: For example, several missions have been launched just to find ice at the lunar poles. Why do astronomers and space agencies care so much about there being ice somewhere?

Answer: 
Ice is very important for future missions. Ice is frozen water so if you send some humans there, you can save a lot of weight by using the water already in the area.

Water is H2O. This means that you can split the Hydrogen and oxygen and then converted it into rocket fuel. H would be the fuel and O the oxidiser.

The final thing is that water means that there could be life. Scientists already suspect that there might be bacteria on Europa (a moon of Jupiter). That would have great scientific purposes.
Water is the big one.

Why?

Because it's made up of Hydrogen and Oxygen, which in turn are the primary materials needed for ongoing spaceflight.

Hydrogen/Oxygen can be liquefied and used as rocket fuel.

We need oxygen to breathe, water to drink, and you can use both to grow more food.

So as long as you have water and a way to generate power, you have all the raw materials you need to operate a space base. Or a refueling station. Or whatever you want.