The sunstone can only hold solar affinity magic and the spell will gather all magic that it can, and because only solar magic will enter the gem the other magic will just stay around it or more likely be absorbed by my other gems in my apartment, and the neutral magic won't decrease or increase as it can't outside of a gem.