Google Vizier: A Service for Black-Box Optimization
Okay, this is awesome (and easy to miss with a just a cursory skim):"5.3    Delicious Chocolate Chip CookiesVizier is also used to solve complex black–box optimization
problems arising from physical design or logistical problems.
Here we present an example that highlights some additional
capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes...
We provided recipes to contractors responsible for providing desserts for Google employees. The head chefs among
the contractors were given discretion to alter parameters if
(and only if) they strongly believed it to be necessary, but
would carefully note what alterations were made. The cookies
were baked, and distributed to the cafes for taste–testing.
Cafe goers tasted the cookies and provided feedback via a
survey. Survey results were aggregated and the results were
sent back to Vizier. The “machine learning cookies” were
provided about twice a week over several weeks...The cookies improved significantly over time; later rounds
were extremely well-rated and, in the authors’ opinions, delicious."