Poo left behind by the creatures is turning the white marble walls green!
The problem's been caused by insects breeding in a nearby polluted river - and there's been a rise in the amount of them.
Cleaners scrub the Taj Mahal's walls every day, but there are worries their elbow grease could ruin the beautiful marble and mosaics.
Authorities are looking for a way to solve the problem as quickly as possible.
"Officials have been asked to investigate as why there is sudden increase in the number of these insects and how to control their population," local official Navneet Sehgal said.
The insects are a type of fly that look like a mosquito.
The incredible building is actually a huge tomb built by Mughal Emperor Shah Jahan for his wife in the 1600s.
It is visited by millions of people every year, including royalty.

Anti-bullying charity, Bullies Out wants a true understanding of what is going on.
Currently, schools must record all racist incidents but there is no legal requirement for them to record other types of bullying.
The Welsh Government said it was in the process of reviewing its policy "to make sure it is as strong as possible".
Bullies Out founder Linda James said: "We need to have that consistent definition of what exactly bullying is.
"We know that some schools are really proactive with their anti-bullying and they deal with it - but we also hear the stories that others are not.
"I think if there was some consistent reporting and recording procedure for all schools then maybe that would make it a lot easier for the schools in general."
Earlier this month, the funeral of 14-year-old Nyah James from Swansea was held. Her mother, Dominique Williams claims she took her own life as a result of being bullied at school.
She said: "It makes us feel terrible because we weren't there to stop it. To find out she was being bullied and going through such pain of her own and we didn't know nothing about it - that kills us every day."
Ms Williams added: "We feel like we've failed her. I think to myself, did I fail her as a mum because she wasn't able to come and talk to me - thoughts like that go through your head."
A South Wales Police spokesperson said officers were still investigating the matter.
In Northern Ireland, legislation making it a legal requirement for schools to record all reports of bullying was given Royal Assent last year and Bullies Out wants similar legislation in Wales.
Rob Williams, director of policy at the National Association of Head Teachers Cymru, said the call to record all bullying incidents was "understandable".
But he warned there was a danger that more focus will be put on the recording process and "not on the effective use of the data".
"Utilising the information to safeguard children is the key part here, and how this translates into positive action," he said.
A Welsh Government spokesman said schools must record all racist incidents - including racial bullying, but there is no legal requirement for them to record other types of bullying.
He added: "Our guidance - Respecting Others - sets out guidance and practical solutions on preventing and responding to incidents of bullying in schools.
"We are in the process of reviewing this policy to make sure it is as strong as possible."
Swarms of insects are using India's top tourist attraction, the Taj Mahal, as a toilet.
[NEXT_CONCEPT]
There are calls for schools in Wales to be legally required to record all incidents of bullying.