The boats were caught in strong winds and torrential rain as they sailed across Lake Togo, about 40km (25 miles) east of the capital, Lome.
Police say the victims were returning home from a funeral on the other side of the lake.
Local official Akouete Edan said there were 21 survivors and that most of those killed were students.
It is not known how many people were on board the boats, and a search is continuing for further bodies.
"When the winds hit, the largest boat carrying some men and drums capsized first," a survivor told a local radio station, according to the Reuters news agency.
"Then two or three other smaller boats carrying women and children also capsized," he said, adding that he had lost three of his children who were with him.
Women's Minister Henriette Amedjogbe travelled with the prime minister to the village of Agbodankope, where many of those killed came from.
"It is a real tragedy," she said, according to the AFP news agency.
"We came to bring you the support of the government and the Togolese people."
At least 36 people have died in Togo when their boats capsized during a storm, officials say.