Farid Khan was stripped naked by the crowd, beaten and dragged through streets before being hanged in Dimapur, the main city in Nagaland state.
Police officers opened fire to try to stop the mob, wounding several people.
Tensions in the country are high following the government's decision to ban India's Daughter, a film about the 2012 gang rape of a student.
The Hindustan Times newspaper reports that the crowd "tore down two gates and took custody" of the suspect, before dragging him to the town's landmark clock tower.
Police say the man was a Bengali-speaking Muslim trader from neighbouring Assam state. He was arrested in February on charges of rape.
There have been recurring tensions in some parts of north-eastern India between Bengali speakers, accused of being immigrants from neighbouring Bangladesh, and local ethnic groups.
Local groups began protests on Wednesday demanding action against the alleged rapist.
Vigilante justice is not unheard of in India but it is rarely seen on this scale. A curfew has been imposed in Dimapur following incidents of arson in some parts of the city.
India's rape crisis has been pushed back to the forefront of public discourse by the decision of the government to ban the BBC documentary India's Daughter, which examines the 2012 gang rape of a young student in Delhi.
The documentary features an interview with one of the men sentenced to death for the attack.
His lack of remorse and suggestions that the victim might have survived if she had not resisted has drawn international outrage and sparked protests across India.
The film was broadcast in the UK on Wednesday night.
A mob of thousands of people lynched a suspected rapist after breaking into a prison in north-east India, police say.