Why is Toradora's ED about oranges? This site has Toradora's second ending theme, Orange (オレンジ), together with lyrics both in Japanese and translated.

First, it's worth noting that the word Orange is ambiguous in English, referring both to a color and a fruit, and it is used to this effect in the song. At one point in the song it talks about a fruit which is turning orange, and at another it talks about an orange. I don't know if we can conclude that these are the same fruit, so if you want to claim that they are actually different fruits feel free to do so.

What is the significance of this fruit, and the story told in the song in general, in the context of the anime? It seems to be symbolic somehow, but I don't know what it's referring to when the singer is talking about an "orange ...[which] was sour, ... [but they] ate it all anyway." 

Can anyone decode the meaning of this song, and what the orange is symbolizing, in the context of the anime?
This song, being sung by the 3 main heroines, was especially made for the anime. The song uses the metaphor of an orange, likening the girls to an unripen, not yet matured, orange.
This blog entry offers a much in-depth look into the meaning behind the song. Though the translations may differ between people, the general meaning stays intact. The blog article mostly compares the CoalGuys and the {Words of Songs} fansubs, with the author trusting the {Words of Songs} version over the CoalGuys fansub (though he admits he likes the fansub lyrics better).
One can assume one of two things from the lyrics: the first being that it's Taiga who is the main voice narrating the lyrics, or the second, that everybody is narrating these lyrics. The following explanations will try to look at the lyrics from both perspectives.

“It’s just that I don’t know/My true self.” This line can easily be seen to be Taiga, or any of the characters. I think Toradora is special in the fact that all the characters can fit into this. It implies how all the characters have a hidden side to themselves that even they may not know.
“Into the broad earth, a single seed/Extended its roots/And bore fruit that was still unripe.” The fruit, first of all, probably symbolizes the narrator(s). This “unripe fruit” thing, I’m guessing, is a lack of maturity for the fruit and the person.
“The fruit that wants to turn orange as soon as possible.” An unripe orange is usually green, while a ripe one is... well, orange (big surprise, huh).  Basically, the person(s) want(s) to mature. This may be because if they mature, they may learn more about themselves in the process.
“by basking in your light.” I think that this light is the object of the narrator’s affection. It’s easier to see that when looking at the next reference to light: “I didn’t want to get hurt/So I fled/When I did that, not even light/Would shine upon me.” Because love is known to be painful, the narrator tries to flee from it, but realizes she cannot grow as a person without love.
“I tried to eat/An orange today, too/But it was still sour and I cried/It reminded me of myself, so I couldn’t throw away the rest.” We can see how the person and the fruit are symbols of each other, and their immaturity is also accented. If we look at the fansub version we see that eating the immature/unripe fruit shows that the person is also not mature yet, while it’s a lot harder to use the {Words of Songs} one.