Answers.Body: <p>According to your description, you already know your function <span class="math-container">$f$</span> to be optimized. So you should use it directly instead of the standard loss functions. In this <a href="https://stackoverflow.com/questions/43818584/custom-loss-function-in-keras">other post</a> there is an explanation of how to use <span class="math-container">$f$</span> as a custom loss function in Keras.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-21T00:32:28.807Z
Answers.LastActivityDate: 2019-11-21T00:32:28.807Z
Answers.UserDisplayName: Diego Gomez
Answers.Body: <p>From your question, it sounds like your only training data is {1,…,} and the network has to magically come up with values {y1,…,y} such that an unknown function is minimized. How do you plan to give feedback to the network during training?</p>
  <p>Your situation appears to be something like this:</p>
  <p><code>X--&gt;Model--&gt;Y--&gt;f(X,Y)</code></p>
  <p>where X is being copied from the first layer to the last layer using a non-sequential architecture.</p>
  <p>The solution to this problem would be to add an extra layer to your network that implements f(X, Y). However, this will only be trainable using standard methods like gradient descent if f(X, Y) is differentiable. If f(X, Y) is not differentiable, then you may need to use a different optimization method for learning the weights of the model and it may be more difficult. Particle swarm optimization is one possibility here.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-14T06:21:29.353Z
Answers.LastActivityDate: 2021-10-02T19:00:29.720Z
Answers.LastEditDate: 2021-10-02T19:00:29.720Z
Answers.UserDisplayName: Mike NZ
Body: <p>I am looking to build a neural network that takes an input vector <span class="math-container">$\mathbf{x}$</span> and outputs a vector <span class="math-container">$\mathbf{y}$</span> such at <span class="math-container">$f(\mathbf{x}, \mathbf{y})$</span> is minimized, where <span class="math-container">$f$</span> is some function. The network will see many different <span class="math-container">$\mathbf{x}$</span> during training to adjust its weights and biases; then I will test the network by using the test set <span class="math-container">$\{\mathbf{x}_1, \dots, \mathbf{x}_n \}$</span> to calculate <span class="math-container">$\sum(f(\mathbf{x}_1, \mathbf{y}), \dots, f(\mathbf{x}_n, \mathbf{y}))$</span> to see if this sum is minimized.</p>
  <p>However, I have no labels for the output <span class="math-container">$\mathbf{y}$</span>. The loss function I am trying to minimize is based on the input and output, instead of the output and label.</p>
  <p>I tried many standard Keras and TensorFlow loss functions, but they are unable to do the job. Any thoughts on how this might be achieved?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-20T23:47:22.607Z
Comments.Text: Can you give the specific task you're working on to add clarity?
Comments.UserDisplayName: mshlis
Tags: <neural-networks><optimization><unsupervised-learning>
Title: Unsupervised learning to optimize a function of the input
UserDisplayName: Y.Z.
ThreadText: # Unsupervised learning to optimize a function of the input
  
  (Asked by: username_0 on 2019-11-20)
  
  I am looking to build a neural network that takes an input vector $\mathbf{x}$ and outputs a vector $\mathbf{y}$ such at $f(\mathbf{x}, \mathbf{y})$ is minimized, where $f$ is some function. The network will see many different $\mathbf{x}$ during training to adjust its weights and biases; then I will test the network by using the test set $\{\mathbf{x}\_1, \dots, \mathbf{x}\_n \}$ to calculate $\sum(f(\mathbf{x}\_1, \mathbf{y}), \dots, f(\mathbf{x}\_n, \mathbf{y}))$ to see if this sum is minimized.
  
  However, I have no labels for the output $\mathbf{y}$. The loss function I am trying to minimize is based on the input and output, instead of the output and label.
  
  I tried many standard Keras and TensorFlow loss functions, but they are unable to do the job. Any thoughts on how this might be achieved?
  
  **Tags:** neural-networks, optimization, unsupervised-learning
  > **Comments:**
  > **username_1**: Can you give the specific task you're working on to add clarity?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-11-21. Score: 1
  
  According to your description, you already know your function $f$ to be optimized. So you should use it directly instead of the standard loss functions. In this [other post](https://stackoverflow.com/questions/43818584/custom-loss-function-in-keras) there is an explanation of how to use $f$ as a custom loss function in Keras.
  
  
  ---
  
  ### Answer by username_3 on 2021-04-14. Score: 0
  
  From your question, it sounds like your only training data is {1,…,} and the network has to magically come up with values {y1,…,y} such that an unknown function is minimized. How do you plan to give feedback to the network during training?
  
  Your situation appears to be something like this:
  
  `X-->Model-->Y-->f(X,Y)`
  
  where X is being copied from the first layer to the last layer using a non-sequential architecture.
  
  The solution to this problem would be to add an extra layer to your network that implements f(X, Y). However, this will only be trainable using standard methods like gradient descent if f(X, Y) is differentiable. If f(X, Y) is not differentiable, then you may need to use a different optimization method for learning the weights of the model and it may be more difficult. Particle swarm optimization is one possibility here.
  
  
  ---
  
  