Thirty-five job centre staff are to lose their jobs in Powys as part of the UK-wide civil service cutbacks announced last year.
Civil service staff protested after last year's announcement
The workers in Newtown are taking "voluntary early release", said the Department for Work and Pensions (DWP).
Union chiefs said the centre was one of 60 in Wales that would be affected by the cuts.
Chancellor Gordon Brown outlined plans to axe more than 104,000 civil service posts in July last year.
Jobcentre Plus, part of the DWP, said Newtown's centre would not close.
A spokeswoman said: "The government is looking at how best to modernise the delivery of its services.
"Newtown Jobcentre will still be open to its customers to give advice about employment, but will no longer undertake benefit processing, which can be more efficiently undertaken in benefit processing centres across Wales.
"It continues to be the case that compulsory redundancy will only be used as a last resort.
"At Newtown Jobcentre 35 staff have volunteered to take voluntary early release."
Darren Williams, of the Public and Commercial Services (PCS) union in Wales, said officials were concerned about the job losses.
"The Department for Work and Pensions is the largest part of the civil service affected with 40,000 job losses in the UK, although 10,000 of those jobs will be redeployed.
"In Wales, 8,000 work for the department and we suspect 2,000 will lose their jobs by 2008, although this has not been confirmed," Mr Williams added.
"The impact to local services will be significant and in areas like Newtown, many people are reliant on these type of relatively high-paid jobs."