Answers.Body: <p>It depends on the complexity of your problem. <span class="math-container">$\mathbb{R}^2 \rightarrow \mathbb{R}^1$</span> looks simple, but I can give you some nonsense complicated examples that need a deep network. So, the complexity of the problem sets the number of layers and neurons. The kind of problem will determine the architecture of your network (if it needs memory or not). In most cases, the <em>mean squared error</em> is ok. However, for the activation function, I would go with the ReLU.</p>
  <p>If the SVM is good enough for your problem, then go with it. Your question is general and an exact answer needs more information about the problem.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-15T21:51:12.787Z
Answers.Text: Previously, you had written that the ReLU is better. I removed the "better", but maybe you should explain why you're suggesting the ReLU. Maybe you should also mention the universal approximation theorems, given the generality of the question, if you heard of them.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-15T15:51:56.610Z
Answers.LastActivityDate: 2020-12-15T21:50:21.317Z
Answers.LastEditDate: 2020-12-15T21:50:21.317Z
Answers.UserDisplayName: amin
Body: <p>Suppose we want to estimate a continuous function <span class="math-container">$f:\mathbb R^2 \rightarrow \mathbb R$</span> based on a sample using a NN (around 1000 examples). This function is not bounded. Which architecture would you choose ? How many layers/neurons ? Which activation functions ? Which loss function ?</p>
  <p>Intuitively, I would go with one hidden layer, 2 neurons, <span class="math-container">$L^2$</span> loss, and maybe the Bent identity for the output and a sigmoid in the hidden layer ?</p>
  <p>What are the advantages of doing something &quot;fancier&quot; than that ?</p>
  <p>Would you also have chosen to use a NN for this job or would you have considered a regression SVM for example or something else (knowing that precision is the goal)?</p>
  
Tags: <neural-networks><objective-functions><activation-functions><regression>
Title: Which NN would you choose to estimate a continuous function $f:\mathbb R^2 \rightarrow \mathbb R$?
UserDisplayName: W. Volante
ThreadText: # Which NN would you choose to estimate a continuous function $f:\mathbb R^2 \rightarrow \mathbb R$?
  
  (Asked by: username_0 on 2020-12-14)
  
  Suppose we want to estimate a continuous function $f:\mathbb R^2 \rightarrow \mathbb R$ based on a sample using a NN (around 1000 examples). This function is not bounded. Which architecture would you choose ? How many layers/neurons ? Which activation functions ? Which loss function ?
  
  Intuitively, I would go with one hidden layer, 2 neurons, $L^2$ loss, and maybe the Bent identity for the output and a sigmoid in the hidden layer ?
  
  What are the advantages of doing something "fancier" than that ?
  
  Would you also have chosen to use a NN for this job or would you have considered a regression SVM for example or something else (knowing that precision is the goal)?
  
  **Tags:** neural-networks, objective-functions, activation-functions, regression
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-12-15. Score: 1
  
  It depends on the complexity of your problem. $\mathbb{R}^2 \rightarrow \mathbb{R}^1$ looks simple, but I can give you some nonsense complicated examples that need a deep network. So, the complexity of the problem sets the number of layers and neurons. The kind of problem will determine the architecture of your network (if it needs memory or not). In most cases, the *mean squared error* is ok. However, for the activation function, I would go with the ReLU.
  
  If the SVM is good enough for your problem, then go with it. Your question is general and an exact answer needs more information about the problem.
  
  > **Comments:**
  > **username_2**: Previously, you had written that the ReLU is better. I removed the "better", but maybe you should explain why you're suggesting the ReLU. Maybe you should also mention the universal approximation theorems, given the generality of the question, if you heard of them.  
  
  ---
  
  