Why must we not wear a hat or cap when listen or teach Dhamma? I read that we must not listen or teach Dhamma if we've wear hat or cap. 

So why we must not wear hat or cap while listen or teach Dhamma even at anniversaries?
This rule has very archaic origins. When listening to a monk explaining the dhamma we normally put the monk in a more respected position (e.g. sitting in a chair while we sit on the floor); when we do this we tend to treat the monk's words as more important so being humble is helping us to learn the dhamma.

At the time of the Buddha sunshades and umbrellas were a sign of luxury. At that time if someone used a sunshade while the monk didn't have one then it is as if they see themselves as more important than the monk and they are less inclined to care about the monk's words.

Thanissaro Bhikkhu's book The Buddhist Monastic Code I has some explaination about this rule


  SN 6.2 records that the Buddha himself had the highest respect for the Dhamma
  he had discovered; that, as others might live under the guidance of a teacher,
  honoring and revering him, the Buddha lived under, honored, and revered the
  Dhamma. He enjoined his followers to show the same respect for the Dhamma
  not only when listening to it but also when teaching it, by refusing to teach it to a
  person who shows disrespect.
  The following set of rules deals with situations in which a listener, in terms of
  the etiquette at that time, would be regarded as showing disrespect for a teacher
  or his teaching. As the Vinaya-mukha notes, a few of these cases—such as those
  concerning footwear—are not considered disrespectful under certain
  circumstances at present, although here the exceptions given for listeners who
  are ill might be stretched to cover any situation where the listener would feel
  inconvenienced or awkward if asked to comply with the etiquette of the
  Buddha’s time. On the other hand, there are many ways of showing disrespect at
  present that are not covered by these rules, and an argument could be made,
  reasoning from the Great Standards, that a bhikkhu should not teach Dhamma
  to a person who showed disrespect in any way.
  Dhamma here is defined as any statement spoken by the Buddha, his disciples,
  seers, or devat›s, connected with the teaching or with its goal. See Pc 7 for a
  more detailed discussion of this point.