On Sunday, the deep-water detector, or towfish, "collided with a mud volcano which rises 2,200 metres from the seafloor," an official statement said.
Both the device and 4,500 metres of snapped cable are now resting on the sea floor.
The search team believe they will be able to recover both at a later date.
The Joint Agency Coordination Centre (JACC), overseeing the search, did not say whether the loss would delay their operation, which they have said will be completed by June.
Flight MH370 disappeared in March 2014, with 239 people on board, during a flight from Kuala Lumpur to Beijing. Based on satellite communications data, the plane is thought to have crashed in the Indian Ocean, but only one piece of debris from the plane has ever been found, on the French island of Reunion.
Search ship Fugro Discovery is now returning to port in Fremantle, Australia, to have a replacement cable fitted, which it will then use with a spare towfish it has on board.
The search is focused on a 120,000-sq-km (46,330-sq-mile) area of the southern Indian ocean. The sonar detector is pulled through the water about 100 metres (330 feet) above the seabed, mapping the underwater terrain.
On Saturday, another piece of possible aircraft debris was found off the coast of southern Thailand, although experts and officials have cast doubt on the idea it might belong to MH370, as prevailing currents would be unlikely to carry debris there from the southern India Ocean.
The Australian team looking for missing Malaysia Airlines flight MH370 has lost the sonar detector they were using for their search.