The process of converting that magic can take endless forms. I'm not sure why but I think it's some combination of belief from sentient beings or the odd and vague rules of magic. However it happens, when magic becomes a certain affinity it slowly fades back into neutral magic or is used up by something in the environment.