Answers.Body: <p>If your black-box function is a differentiable function, then you can find an optimum solution with only one trainable vector.</p>
  <p>Suppose you have the 250 length vector <span class="math-container">$V$</span>, use the binary step activation function (you can use sigmoid), and use <span class="math-container">$f(x)$</span> as your loss function (if you want to maximize the output use <span class="math-container">$\frac{1}{f(x)}$</span>). And then learn the <span class="math-container">$V$</span> by backpropagation.</p>
  <p>If your function is not differentiable I don't see any solution for that.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-26T23:32:19.087Z
Answers.Text: Do you mean that even if unknown function is un differential, 250 length vector/kernel will give approximate result?
Answers.UserDisplayName: Cloud Cho
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-26T23:34:46.853Z
Answers.Text: I said it should be differentiable.
Answers.UserDisplayName: Peyman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-29T21:31:06.293Z
Answers.LastActivityDate: 2023-09-26T23:33:52.710Z
Answers.LastEditDate: 2023-09-26T23:33:52.710Z
Answers.UserDisplayName: Peyman
Body: <p>I have a non-linear black box function, which inputs a vector(size=250) and outputs a scalar value;     <code>f(x) = value</code>.</p>
  <p>The x variable is a vector of size 250 and has binary elements, e.g.: <code>x = [0, 1, 1, 1, 0, 0, ...]</code></p>
  <p>The result is just a scalar value and I am trying to maximize this value via a binary differential evolution algorithm. Additionally, the calculation time of <code>f(x)</code> is about 10 seconds.</p>
  <p>Since there are <span class="math-container">$2^{250}$</span> different x vectors, it is really hard to find the optimizer(x) of this function. However, a result that is not the optimum but not close to it would be also still an acceptable answer.</p>
  <p>I was thinking, if I am using the right approach here in the binary differential evolution algorithm, I would appreciate it if you would give me your comments or feedback!</p>
  <ul>
  <li>Find some meaningful starting population (size=100) for the binary evo.</li>
  <li>Run f(x) with this population</li>
  <li>Create a surrogate model (probably a simple neural network that maps the vector to the result values)</li>
  <li>Run the surrogate model 10000 times with a random selection of x</li>
  <li>Select the best-performing 100 xs from the 10000 sample</li>
  <li>Initialize the differential evolution with the best-performing 100 surrogate model x</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-26T23:29:49.457Z
Comments.Text: Curve Fitting (https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html)?
Comments.UserDisplayName: Cloud Cho
Tags: <machine-learning><optimization><non-linear-regression><black-box>
Title: Optimizing a blackbox function with binary states
UserDisplayName: oakca
ThreadText: # Optimizing a blackbox function with binary states
  
  (Asked by: username_0 on 2023-04-24)
  
  I have a non-linear black box function, which inputs a vector(size=250) and outputs a scalar value; `f(x) = value`.
  
  The x variable is a vector of size 250 and has binary elements, e.g.: `x = [0, 1, 1, 1, 0, 0, ...]`
  
  The result is just a scalar value and I am trying to maximize this value via a binary differential evolution algorithm. Additionally, the calculation time of `f(x)` is about 10 seconds.
  
  Since there are $2^{250}$ different x vectors, it is really hard to find the optimizer(x) of this function. However, a result that is not the optimum but not close to it would be also still an acceptable answer.
  
  I was thinking, if I am using the right approach here in the binary differential evolution algorithm, I would appreciate it if you would give me your comments or feedback!
  
  * Find some meaningful starting population (size=100) for the binary evo.
  * Run f(x) with this population
  * Create a surrogate model (probably a simple neural network that maps the vector to the result values)
  * Run the surrogate model 10000 times with a random selection of x
  * Select the best-performing 100 xs from the 10000 sample
  * Initialize the differential evolution with the best-performing 100 surrogate model x
  
  **Tags:** machine-learning, optimization, non-linear-regression, black-box
  > **Comments:**
  > **username_1**: Curve Fitting (https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html)?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-29. Score: 0
  
  If your black-box function is a differentiable function, then you can find an optimum solution with only one trainable vector.
  
  Suppose you have the 250 length vector $V$, use the binary step activation function (you can use sigmoid), and use $f(x)$ as your loss function (if you want to maximize the output use $\frac{1}{f(x)}$). And then learn the $V$ by backpropagation.
  
  If your function is not differentiable I don't see any solution for that.
  
  > **Comments:**
  > **username_1**: Do you mean that even if unknown function is un differential, 250 length vector/kernel will give approximate result?  
  > **username_2**: I said it should be differentiable.  
  
  ---
  
  