SYDNEY (Reuters) - About 20 asylum seekers on Friday left an Australian-run detention center due to close down in Papua New Guinea, but hundreds of holdouts faced forcible eviction amid an immigration standoff that has blighted Australia for years. Nearly 600 men have barricaded themselves into the Manus island center for more than 10 days without regular food or water, defying closure attempts by Australia and Papua New Guinea in what the United Nations calls a  looming humanitarian crisis . The deadline to move is Saturday. The asylum seekers fear reprisals if they move to transit centers pending possible resettlement in the United States. The main camp was closed on Oct. 31 and water and power have been cut off. Some locals are angry at what they perceive as preferential treatment for the asylum seekers, many of them well educated, in a poor, rural society, and some of the detainees have come under attack when on release from the camp.  Bowing to pressure from Papua New Guinea, which said it would forcibly evict and  apprehend  any holdouts on Saturday, around 20 men left the camp for one of the transit centers, three asylum seekers told Reuters. The rest vowed to stay put, setting the scene for a possible clash.  Volker Turk, assistant high commissioner for protection at U.N. refugee agency UNHCR, called on both governments to exercise restraint and not to use violence, taking into account people who had been in the processing center for years.  They re in a very vulnerable state with not much hope in sight,  he said.  We have been visiting Manus island several times over the last couple of years, we have reported on the very dire conditions in these centers. It s now really high time to bring an end to this unconscionable human suffering.   In Sydney, about 200 protesters picketed a fundraiser for the ruling Liberal Party, heckling arriving guests and demanding that the men be allowed to settle on the mainland. In Melbourne, more than 1,000 people held similar protests, the Australian Associated Press news agency said.   Australia has used the center, and a camp on the tiny Pacific island of Nauru, to detain asylum seekers who try to reach its shores by boat. It says boat arrivals will never enter Australia, even if found to be refugees, as this would encourage people smugglers in Asia. Australia says the policy prevents people drowning at sea, but it has been widely condemned.  We aren t going to be stood over by people that have better accommodation to move to,  Immigration Minister Peter Dutton said in a radio interview. The U.N. Human Rights Committee urged Australia to bring its migration laws into line with world standards. It is concerned about conditions in Manus and on Nauru, citing safety fears and instances of assault, sexual abuse, self-harm and suspicious deaths. The asylum seekers, drawn largely from Afghanistan, Iran, Myanmar, Pakistan, Sri Lanka, and Syria, use rubbish bins to collect rainwater and set up makeshift shelters to protect against the tropical sun and rain. Papua New Guinea police returned to dismantle parts of the camp on Friday, large parts of its fence having been removed the previous day.  They are destroying our shelters,  said Behrouz Boochani, a Kurdish journalist from Iran, detained for more than four years.  They destroyed the rubbish bins where we have been collecting water too.   