Answers.Body: <p>A first observation is that <span class="math-container">$f(x)$</span> is shared across all the <span class="math-container">$n$</span> dimensions of <span class="math-container">$x$</span> so you can approximate it using a fully connected neural network with a 1-d output. You can think of it as a block in your architecture. To get a variable number of <span class="math-container">$f(x_i)$</span> you just run the whole network on each <span class="math-container">$x_i$</span> up to <span class="math-container">$k$</span>. I expect this won't be hard to parallelize.</p>
  <p>The other function <span class="math-container">$G$</span> doesn't really need to know anything about <span class="math-container">$k$</span>, it just requires the sum of <span class="math-container">$f(x_i)$</span> which is nothing but a scalar. Hence, to approximate <span class="math-container">$G$</span> you just need another fully connected neural network with 1-d input and 1-d output layers.</p>
  <p>As for how big the middle layers should be, this depends on many factors and foremost on your data and how complex the functions <span class="math-container">$f$</span> and <span class="math-container">$G$</span> are. You'd better experiment a bit on both cases.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-05T15:27:52.297Z
Answers.LastActivityDate: 2022-07-05T16:34:51.977Z
Answers.LastEditDate: 2022-07-05T16:34:51.977Z
Answers.UserDisplayName: sfotiadis
Body: <p>I have a function <span class="math-container">$g$</span> that takes a vector <span class="math-container">$x$</span> of size <span class="math-container">$n$</span> and an integer <span class="math-container">$k$</span> in <span class="math-container">$1, \ldots, n$</span>. I know this function is of the form
  <span class="math-container">$$g(x,k) = G\left(\sum_{i=1}^k f(x_{i})\right),$$</span> where <span class="math-container">$f$</span> and <span class="math-container">$G$</span> are some unknown functions.</p>
  <p>I want to train a neural network that estimates the function <span class="math-container">$g$</span>. What is the best way to proceed given the structural information I have about the function?</p>
  <p>Of course, I can train the neural network without taking the specific structure into account, but I believe this is not the best way. I can also estimate <span class="math-container">$n$</span> neural networks for <span class="math-container">$g(x,1), \ldots, g(x,n)$</span>, but I guess this is not the best way either. Is there a good / standard way to proceed?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-05T15:08:57.910Z
Comments.Text: Do you have any information about f and/or G?
Comments.UserDisplayName: sfotiadis
Tags: <neural-networks><deep-neural-networks><supervised-learning>
Title: How to use information on a function to design a neural network learning that function?
UserDisplayName: Aristodog
ThreadText: # How to use information on a function to design a neural network learning that function?
  
  (Asked by: username_0 on 2022-07-05)
  
  I have a function $g$ that takes a vector $x$ of size $n$ and an integer $k$ in $1, \ldots, n$. I know this function is of the form
  $$g(x,k) = G\left(\sum\_{i=1}^k f(x\_{i})\right),$$ where $f$ and $G$ are some unknown functions.
  
  I want to train a neural network that estimates the function $g$. What is the best way to proceed given the structural information I have about the function?
  
  Of course, I can train the neural network without taking the specific structure into account, but I believe this is not the best way. I can also estimate $n$ neural networks for $g(x,1), \ldots, g(x,n)$, but I guess this is not the best way either. Is there a good / standard way to proceed?
  
  **Tags:** neural-networks, deep-neural-networks, supervised-learning
  
  > **Comments:**
  > **username_1**: Do you have any information about f and/or G?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-05. Score: 0
  
  A first observation is that $f(x)$ is shared across all the $n$ dimensions of $x$ so you can approximate it using a fully connected neural network with a 1-d output. You can think of it as a block in your architecture. To get a variable number of $f(x\_i)$ you just run the whole network on each $x\_i$ up to $k$. I expect this won't be hard to parallelize.
  
  The other function $G$ doesn't really need to know anything about $k$, it just requires the sum of $f(x\_i)$ which is nothing but a scalar. Hence, to approximate $G$ you just need another fully connected neural network with 1-d input and 1-d output layers.
  
  As for how big the middle layers should be, this depends on many factors and foremost on your data and how complex the functions $f$ and $G$ are. You'd better experiment a bit on both cases.
  
  
  ---
  
  