Question: How do we know that dark matter is dark, in the sense that it doesn't give out any light or absorb any? It is impossible for humans to be watching every single wavelength. For example, what about wavelengths that are too big to detect on Earth?

Answer: 
There is indeed very good reason to believe dark matter is dark - apart from all the evidence from "missing mass" in luminosity counts and gravitational lensing studies.

This comes from theories of large-scale structure formation:
That there has always had to be some sort of matter that doesn't interact electromagnetically at all is crucial to most scenarios of large-scale structure formation. The density fluctuations in the present universe would be too large than what would be predicted if there were only ordinary baryonic matter that interacted only electromagnetically. With dark matter, you can have something that gravitates yet decouples from radiation much before baryonic matter does. This allows the dark matter to form gravitational wells (under collapse) which have a much longer time to expand with the universe. By the time ordinary matter decouples from radiation and joins the rest of the expansion flow, the ordinary matter will quickly fall into these large gravitational wells of the dark matter that have had far more time to grow. This, in a way, amplifies density perturbations in the early universe and allows large-scale structure to form. (to the extent that we see it today in the form of clusters and galaxies)

The required amount of dark matter calculated, in this way, in order to observe the present scale of density fluctuations matches very well with the amount of dark matter required to explain galactic rotation curves, gravitational lensing, etc. So there's excellent agreement that all of these are due to the same thing - some sort of matter which doesn't interact electromagnetically at all, viz. dark matter.
Dark matter was originally called that because it was matter.... that was dark. In other words, it was inferred that there must be something out there, in galaxies and in clusters of galaxies, that had a gravitational effect, but which could not be readily detected using electromagnetic waves at any wavelength.

Now initially it was thought that this "dark matter" could just be stuff that was hard to detect and that had a large mass-to-light ratio. An example would be lost golf balls. Golf balls do not emit very much light - well, not unless you make them very hot. On the other hand they do have mass. If you have enough (cold) golf balls then it could be that all those lost golf balls explain why galaxies rotate too fast as a function of radius or why the galaxy velocity dispersion in galaxy clusters seems to high for the amount of luminous matter that can be accounted for. More seriously, the cosmic lost golf balls that have been considered were very faint stars, cold brown dwarfs, old white dwarfs or perhaps lots and lots of planets or mini-black holes.

Pretty much all of these possibilities have now been excluded (primordial black holes may still be a possibility) using, for example, transit and microlensing experiments.

At the same time, what we mean by "dark matter" has evolved to mean *non-baryonic* dark matter. This is not just matter which is faint, but matter which does not interact electromagnetically at all. Such matter is by definition, absolutely dark.

Why is such a material proposed? Well firstly there is the failure to detect enough baryonic dark matter (i.e. the lost golf balls which are just very dim), which does interact electromagnetically but is just dim. Secondly, we know how much baryonic matter there is in the universe by estimating the primordial abundances of helium and deuterium produced in the big bang. The amount falls short by a factor of about 6 of the amount of gravitating matter required to explain the dynamics of the universe, galaxies and galaxy clusters. The remaining 5/6 must be non-baryonic and since we can't have charged leptons like electrons without an equal number of positively charged baryons, then it does not interact electromagnetically either (any quantity of positrons would have annihilated long ago). It is therefore, by definition, dark at all wavelengths.