The news that his bronze medal was being taken away by the Olympic authorities was met with shock and disappointment in Aviemore.
The Highland town was a scene of jubilation when he made his victorious return from the Winter Games last month.
But on Thursday there was disquiet at the International Olympic Committee's decision.
Duncan Rankin, a friend of Baxter, said: "He will be feeling devastated - I've known him all my life and everyone knows that he has never done anything wrong, and he will still always be our hero.
"I know for a fact that he does not deserve to lose a medal and I think the IOC should re-evaluate the doping system."
He added: "It should be more specific regarding the amount of drugs that athletes are supposed to have taken."
Baxter - the first Briton ever to win an Olympic skiing medal - failed a drugs test in Salt Lake City.
He has received an automatic ban from competition skiing which lasts until June.
The International Skiing Federation now has to decide now whether to extend that ban, potentially prolonging it by up to two-years.
But Baxter has maintained the positive drugs test came from an over-the-counter nasal inhaler which he bought in the US.
Aviemore councillor Bob Severn attacked the IOC's findings, and said the dope testing regime was flawed.
A former drug squad officer with Northern Constabulary, he said: "The Olympic Committee should be ashamed of themselves.
"If they can't differentiate between medications and the difference in methamphetamine L and D - methamphetamine L is the medication, methamphetamine D is the one used by drug abusers - there's something far wrong.
"We here in Aviemore know Alain is not a drugs cheat, all the people that race with him, they all know the same, and we'll be here to support him in his hour of need as we were in his hour of glory."
Councillor Severn added that if the skier needed any financial help with any appeal, the Aviemore community would do what it could to help.
Baxter's grandfather Chic said: "It is a very sad day. We must admit we did expect them to take the medal from him, but it is still devastating.
"We thought they would do it because they don't even take mistakes lightly. But why do this to a young man just for using a decongestant.
"The quantity found in his was so small that there is no doubt in my mind or anyone's mind that he had taken them to enhance his abilities."
Peter Steinle, the owner of the Cairngorm Hotel where some of the celebrations for Baxter were held, said the IOC had taken an unnecessarily harsh line.
He said: "Somebody said to me at the weekend that the actual amount of the substance found in his body would be less than that had he kissed a girl who was high on speed.
"That seems ridiculous to me that he should be penalised in such a way for apparently such a minor offence.
"It's the triviality that people can't get their heads around."