Hey guys, since the question was posted they opened up Modernist Pantry , which specifically caters to this need .
They supply chemicals for molecular gastronomy in consumer-sized portions. For $10 or less apiece, you can order small amounts of emulsifiers, gelling agents, foaming agents, etc.
They also have spherification kits, the Texturas products, and Molecule-R's lineup.
cooking.stackexchange.com/users/6345/BobMcGee
I'm not sure how helpful I can be
but, at least coming from a chemistry lab background, I can say you might run into issues trying to buy food additives in such amount.
Things like potassium chloride are easy to get because they have many uses outside of food and aren't really harmful unless you swallow the whole bottle.
Many compounds are either obscure, hard to make, or toxic in larger amounts.
You can find that many additives are used in very small amounts and if you try and buy them from ie.
a chemical supplier they get rather expensive.
For example the compound you link to as sold by sigma aldrich (one such supplier)
I found here at $32.40 for a lousy 5 grams, and they do not sell to individuals as most chemical suppliers don't.
You will naturally have better luck the more common a given compound is.
With that said you might have success on eBay (though naturally be skeptical of anything you buy, especially stuff you're ingesting!).
I know I have seen quite a few things on there including food-related such as benzoate salts (preservative), saccharin sodium (artifical sweetener), or food dyes.
Don't forget to make sure anything you buy is USP grade (meaning it is certified specifically for food), as I believe that this imposes certain restrictions such as heavy metal content that might not be as restrictive even in "pure" amounts if it's meant for lab use as opposed to general consumption.
Also, you might have success looking up the uses for the additive in question and looking for stores that cater to them.
As a rushed example, I found sodium EDTA available from a photography supplier , which I have seen the mixed sodium/calcium salt listed as an ingredient in some items.
This really depends on how useful the additive is of course!
Maybe someone else can help you from a different perspective.
cooking.stackexchange.com/users/94/Kryptic
See @Kryptic's answer as well.
Just to supplement, if you are going to buy stuff from chemical suppliers, you want food grade chemicals.
Pharmacopaedic grades (i.e. BP, USP, etc.) are also okay. and a bit of background information.
you can get chemicals in different grades from manufacturers, e.g. analytical grade, reagent grade, etc.
these are for lab works and they specify for purity and cross reaction.
the constituency and purity is of utmost importance (e.g. 99.5% - 100% pure).
food (and pharmacopedic, to certain extent) grade specify for (the lack of) harmful chemicals that is active biologically (e.g. heavy metals) and is a little bit lax (e.g. 98%-100%) on actual purity.
cooking.stackexchange.com/users/164/bubu
Khymos has a list of suppliers on his web site organized by region.
Most of those are more geared toward the molecular gastronomy additives (namely E400-E499 ), but some of them may sell other additives as well.
Additionally, if the product you're looking for happens to be in the Texturas collection , you can have a look at Adria's list of distributors .
cooking.stackexchange.com/users/41/Aaronut
Since MSG was explicitly asked about too: Many asian grocers do carry the original Ajinomoto brand MSG, usually comes in transparent plastic bags with red printing on it.
Also look there for food-grade lye (used for making ramen and other alkaline noodles), food colorings, and sometimes flavorings.
Odd gums and fillers (xanthan, inulin) can be found wherever gluten-free baking is catered to.
Check hardware stores that carry canning supplies when it is canning season
- I have seen some unusual additives (preservatives and pH regulators) on offer there which grocers usually do not carry.
A few old-school food chemicals (eg hartshorn, potash) will be offered by stores catering to german style christmas bakery when it is in season.
cooking.stackexchange.com/users/35312/rackandboneman
Have you looked at your local bulk stores?
A new one that doesn't cater to the organic-natural-whatever clientele opened up here.
One of the bins in their spice area? MSG.
Also, in this context, bulk means "Take however much you need out of the bin", not "Buy by the kilogram".
cooking.stackexchange.com/users/269/aehiilrs
I occasionally want a pure food additive for various reasons (pure potassium chloride for sports drinks, pure MSG for everything), but I can't find any decent places to get them except amazon.com. Now I'm looking for  disodium inosinate , and even Amazon doesn't have that. I expect I'll be looking for weird things like this in the future, so I'm wondering where people buy food additives (besides bulk suppliers).
Where do I buy food additives (not in bulk)?
cooking.stackexchange.com
<additives>
cooking.stackexchange.com/questions/679
cooking.stackexchange.com/users/152/Brendan Long
MSG is used in Asian food, some chemicals like hartshorn or potash are used in traditional German baking and some gums and fillers like xanthan and inulin are used in gluten-free cooking. Suppliers serving those markets may stock the products you are looking for. The Modernist Pantry also caters to this need. When sourcing chemicals, ensure that the products are food-grade. There are shops specialising in individual cultural products where you could find certain chemicals (MSG for example). Make sure you are always getting food grade products.
The Modernist Pantry specialises in this type of product. Try specialist shops such as Asian supermarkets for MSG, anywhere stocking German baking ingredients or gluten-free suppliers. Make sure you are getting food grade chemicals.