The organisation says many of those returning to Somalia are motivated by fear of being forced out.
Tens of thousands of refugees have been making the journey back home after years in the camp.
Both Kenya and Somalia say it is time Dadaab was shut down.
But HRW says Kenya is not giving the refugees a real choice between being repatriated or staying.
It also says the UN's refugee arm, the UNHCR, is not giving the refugees accurate information about security conditions in Somalia and they face serious risk of persecution or threats to their lives if they return.
Pressuring refugees to return to a place where their lives or freedom could be under threat is a breach of the 1951 Refugee Convention, the organisation says.
Some of the refugees are also choosing to leave because they fear being deported later and forfeiting a $400 (Â£300) UN cash grant, the organisation says.
"There is no way these returns can be considered voluntary," said HRW's Bill Frelick.
Some refugees who have left Dadaab have since found themselves stranded near the border after regional leaders in Jubaland, which borders Kenya and where most of the returnees are going, stopped receiving them, citing inadequate humanitarian support.
The Dadaab camp in Kenya is home to more than 300,000 Somalis.
It was set up in 1991 to house families fleeing conflict and some people have been living there for more than 20 years.
Kenya says attacks on its soil have been planned in the camp.

Angel Martinez, Sylvan Ebanks-Blake and Dan Jones were announced on Friday.
They are now joined by Liam Graham, Curtis Morrison, Jake Beesley, Thorsten Stuckmann, Lloyd Allinson, Sadiq El Fitouri, Jay O'Shea and Charlie Raglan.
Veteran Ritchie Humphreys, 39, is in talks regarding a coaching role.
"It is never easy having to tell players that they will not be offered new contracts," said manager Gary Caldwell.
"But it is something that has to be done as we look to rebuild over the summer and shape a squad capable of competing in League Two.
"We have made contract offers to four players and hope that they will all sign.
"I would like to thank the players who are leaving the club for their service and wish them all the very best for the future."

The London-based musician beat eight-year-old magician Issy Simpson and comedian Daliso Chaponda to become the winner of the ITV talent show.
The 32-year-old won £250,000 in prize money and an appearance at the Royal Variety Performance.
ITV moved the final of the show forward by a day to avoid a clash with Ariana Grande's benefit concert for the Manchester bombing victims on Sunday.
Myers had previously supported artists such rapper Kanye West.
His performance in the final took in a variety of musical styles and included drumming.
He fell to the floor after presenters Ant and Dec broke the news he had received the most votes from the public.
Judge Simon Cowell said: "As a talent, as a person, Tokio right now, this was the most important thing for him, I'm thrilled for him."
Teenage singer Sarah Ikumu had been chosen as the judges' wildcard choice for the final.
The other acts taking part were: magician Matt Edwards, The Missing People Choir, singing pensioners the Pensionalities, dance act MerseyGirls, comedian Ned Woodman, singer Kyle Tomlinson, and magic double act DNA.
The repatriation of Somali refugees from the Dadaab refugee camp in Kenya fails to meet international standards for voluntary return, a Human Rights Watch report has found.
[NEXT_CONCEPT]
Chesterfield will offer deals to Ian Evatt, Laurence Maguire, Dan Gardner and Rai Simons but have confirmed that 12 players will leave the club following relegation to League Two.
[NEXT_CONCEPT]
Pianist Tokio Myers has been named winner of Britain's Got Talent.