The protesters made the 12km (7-mile) journey to highlight high levels of bacteria in the bay.
They said they were concerned about the health of the athletes taking part in Olympic sailing competitions.
Trials for the events are due to start in Rio waters next Saturday.
The  authorities in Rio initially promised that 80% of the bay would be cleaned up in time for the Games, but they admitted more recently that they would not meet these targets.
Only one of eight water treatment plants has been built so far.
The Brazilian sailor and Olympic medallist Isabel Swan was one of the organisers of the protest.
She  competed in the Beijing Olympics in 2008 and says the area used for sailing in China had problems with a large algae bloom but this was dealt with two months ahead of the competition.
She told the BBC the failure by Rio de Janeiro city and state to meet their commitments made her feel "very bad".
"It's important that we can leave a legacy for our children and for Brazilian people," she said.
"It was very important to bring the Olympic Games to Rio, but we weren't able to clean the bay.
I want to create a movement that will inspire people to defend Guanabara Bay and help the government make a real commitment to cleaning it up. "
Trials for the Olympic sailing competitions due to start next Saturday have been hampered by concerns about the impact on the health of the athletes.
A recent investigation by the Associated Press showed that the waterways in Rio to be used in the Olympics are carrying high counts of disease-causing viruses directly linked to human sewage.
Sergio Ricardo, an ecologist and founder of the Bahia Viva (Living Bay) movement, which organised the protest, said the pollution which affects the bay is caused by a number of different factors.
"The main challenges are sewage treatment and industrial control. The oil industry is encroaching on the fishermen in the bay. We need public policy for fishing, reforestation and sewage treatment so that the bay can live."

It happened at about 03:15 BST on Saturday at a house in Austin Drive, Tandragee.
The petrol bomb was thrown at the front of the house and fire fighters were called to put out the blaze.
A man in his 30s and a woman in her 20s who were in the house at the time escaped injury. The kitchen in the house was smoke damaged.
Police are working to establish a motive for the attack. They have appealed for witnesses.

Councils will decide next week whether to conduct feasibility studies for the radioactive repository.
Protesters from all over the country carried banners for 2.5km (1.5miles) through the valley of Ennerdale to lobby against the plans.
A petition against the proposals now has more than 17,000 signatures.
Unite, which represents staff at the Sellafield plant, gave its backing ahead of a public meeting in Carlisle on Friday.
Allerdale and Copeland borough councils and Cumbria County Council are due to vote on 30 January on whether to proceed to "Stage 4" of the process that could see the repository built.
The proposed facility could be up to four times the size of Sellafield and take 15 years to build.
A flotilla of more than 30 boats and water craft has cruised across Rio de Janeiro's Guanabara Bay to protest against water pollution with a year to go before the 2016 Olympics.
[NEXT_CONCEPT]
A man and woman have escaped injury in a petrol bomb attack in County Armagh.
[NEXT_CONCEPT]
Hundreds of people have taken part in a march against the possibility of building an underground nuclear waste site in Cumbria.