title: Multidimensional hyperparameter search with vw-hypersearch in Vowpal Wabbit
body: <p><a href="https://github.com/JohnLangford/vowpal_wabbit/blob/master/utl/vw-hypersearch" rel="nofollow">vw-hypersearch</a> is the Vowpal Wabbit wrapper intended to optimize hyperparameters in vw models: regularization rates, learning rates and decays, minibatches, bootstrap sizes etc. In the <a href="https://github.com/JohnLangford/vowpal_wabbit/wiki/Using-vw-hypersearch" rel="nofollow">tutorial</a> for vw-hypersearch there is a following example:</p>
  
  <pre><code>vw-hypersearch  1e-10  5e-4  vw  --l1 %  train.dat
  </code></pre>
  
  <p>Here <code>%</code> means the parameter to be optimized, <code>1e-10  5e-4</code> are the lower and upper bounds for the interval over which to search. The library uses <a href="https://en.wikipedia.org/wiki/Golden_section_search" rel="nofollow">golden section search</a> method to minimize the number of iterations. </p>
  
  <p>But what if I want to search over multiple hyperparameters? From the sources like <a href="https://github.com/JohnLangford/vowpal_wabbit/issues/738" rel="nofollow">this</a> github issue discussion, I get a hint that possibly no multidimentional search methods are realized in vw. Thus, the only way out is to write one's own task-specific optimizers. Am I right?</p>