Answers.Body: <p>Even though (as discussed above) this problem is best handled by pre- and post-processing, <strong>yes</strong>, a neural network can be used for this problem.  <em>One possible way</em> to do it is as follows:</p>
  <ol>
  <li>Create your training data.<br />
  The features <code>x_train</code> (<span class="math-container">$x_i\ \in\mathbb{R}^{400}$</span>) will be the input vectors.<br />
  The labels <code>y_train</code> (<span class="math-container">$y_i\ \in\mathbb{R}^{100}$</span>) will be derived by subsetting the input vectors to include every 4th element of <span class="math-container">$x_i$</span> and then setting the max values <span class="math-container">$= 1$</span> and all others <span class="math-container">$= 0$</span>.</li>
  </ol>
  <pre><code># The below (python) code illustrates how this can be done
  
  import numpy as np
  x = np.random.randn(400)
  y = x[::4]
  idx = np.argwhere(y == max(y))
  y = np.zeros_like(y)
  y[idx] = 1
  
  </code></pre>
  <ol start="2">
  <li><p>Architect and train a neural network with 400 input units and 100 output units.  You can try different hyperparameters, starting with simple models first.  The choice of loss function can also be a hyperparameter, but you can try common loss functions like crossentropy loss (CEL) and root mean squared error (RMSE) first.</p>
  </li>
  <li><p>The output units can have a sigmoid activation so that all the values are <strong>between <span class="math-container">$0$</span> and <span class="math-container">$1$</span></strong>.  I'm not sure how well a unit step function would work here as an activation function, but that would guarantee that the values are <strong>either <span class="math-container">$0$</span> or <span class="math-container">$1$</span></strong>.  With the unit step function, there is no need for post-processing.  You can experiment with both.</p>
  </li>
  <li><p>If you use a sigmoid activation, then you will still need to do some post-processing to ensure that the final values in the output vector are <span class="math-container">$0$</span> or <span class="math-container">$1$</span>. For example, you can set values <span class="math-container">$&gt;0.95 = 1$</span> and all others <span class="math-container">$= 0$</span>.</p>
  </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-14T11:09:36.157Z
Answers.LastActivityDate: 2022-11-14T11:42:27.067Z
Answers.LastEditDate: 2022-11-14T11:42:27.067Z
Answers.UserDisplayName: Snehal Patel
Body: <p>I have an input vector of shape say 1 x 400. It's fed into a network that outputs a 1 x 100 vector. I want to design a model that only considers every 4th value of this tensor and gives me the max values.</p>
  <p>For example, a demo tensor could be <code>[0.1, 0.2, 0.4, 3, 0.7, 0.2, 0.6, 8, 0.3, 0.1, 0.5, 12, 0.7, 0.1, 0.9, 12]</code>. For this 1x16 input, the corresponding output would be a 1x4 one. I want the model to look only at values <code>3,8,12,12</code> and then predict an output of the following form <code>0,0,1,1</code>. (note that there could be more than 1 1s in the output)</p>
  <p>I have tried using MLPs to do this using CELoss but to no avail. Can this be solved using some particular architecture/loss only or am I doing something wrong?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-13T20:29:19.993Z
Comments.Text: I don't understand the problem. Why aren't you just creating an input vector with every fourth value?
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-13T23:57:43.563Z
Comments.Text: Every 4th element of $[0.1, 0.2, 0.4, 3, 0.7, 0.2, 0.6, 8, 0.3, 0.1, 0.5, 12, 0.7, 0.1, 0.9, 8]$ would be $[3,8,12,8]$, not $[3,8,12,12]$.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-14T00:04:42.583Z
Comments.Text: I agree with @Brian O'Donnell.  The question is unclear.  It is also unclear why do you need a model to input every 4th element?  This is simply a preprocessing operation.  Second, why do you need a model to return binary coded output of max $1$ vs. not max $0$.?  This can be done with good old fashioned argmax operations, etc.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-14T02:13:10.253Z
Comments.Text: so to summarize, your problem does not required any complicated model, just a few lines of processing code can do.
Comments.UserDisplayName: lpounng
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-14T02:48:18.997Z
Comments.Text: I understand it seems like a preprocessing problem (and it actually is). I was trying to understand if NNs can learn such functions if designed properly, given that I have a lot of training examples of the above form.
Comments.UserDisplayName: helloworld
Tags: <neural-networks><machine-learning><deep-learning><function-approximation>
Title: A practice neural network to find maximum values from subset
UserDisplayName: helloworld
ThreadText: # A practice neural network to find maximum values from subset
  
  (Asked by: username_0 on 2022-11-13)
  
  I have an input vector of shape say 1 x 400. It's fed into a network that outputs a 1 x 100 vector. I want to design a model that only considers every 4th value of this tensor and gives me the max values.
  
  For example, a demo tensor could be `[0.1, 0.2, 0.4, 3, 0.7, 0.2, 0.6, 8, 0.3, 0.1, 0.5, 12, 0.7, 0.1, 0.9, 12]`. For this 1x16 input, the corresponding output would be a 1x4 one. I want the model to look only at values `3,8,12,12` and then predict an output of the following form `0,0,1,1`. (note that there could be more than 1 1s in the output)
  
  I have tried using MLPs to do this using CELoss but to no avail. Can this be solved using some particular architecture/loss only or am I doing something wrong?
  
  **Tags:** neural-networks, machine-learning, deep-learning, function-approximation
  > **Comments:**
  > **username_1**: I don't understand the problem. Why aren't you just creating an input vector with every fourth value?  
  > **username_2**: Every 4th element of $[0.1, 0.2, 0.4, 3, 0.7, 0.2, 0.6, 8, 0.3, 0.1, 0.5, 12, 0.7, 0.1, 0.9, 8]$ would be $[3,8,12,8]$, not $[3,8,12,12]$.  
  > **username_2**: I agree with @Brian O'Donnell.  The question is unclear.  It is also unclear why do you need a model to input every 4th element?  This is simply a preprocessing operation.  Second, why do you need a model to return binary coded output of max $1$ vs. not max $0$.?  This can be done with good old fashioned argmax operations, etc.  
  > **username_3**: so to summarize, your problem does not required any complicated model, just a few lines of processing code can do.  
  > **username_0**: I understand it seems like a preprocessing problem (and it actually is). I was trying to understand if NNs can learn such functions if designed properly, given that I have a lot of training examples of the above form.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-11-14. Score: 1
  
  Even though (as discussed above) this problem is best handled by pre- and post-processing, yes, a neural network can be used for this problem.  One possible way to do it is as follows:
  
  Create your training data.
  The features x_train ($x_i\ \in\mathbb{R}^{400}$) will be the input vectors.
  The labels y_train ($y_i\ \in\mathbb{R}^{100}$) will be derived by subsetting the input vectors to include every 4th element of $x_i$ and then setting the max values $= 1$ and all others $= 0$.
  
  # The below (python) code illustrates how this can be done
  
  import numpy as np
  x = np.random.randn(400)
  y = x[::4]
  idx = np.argwhere(y == max(y))
  y = np.zeros_like(y)
  y[idx] = 1
  
  
  
  Architect and train a neural network with 400 input units and 100 output units.  You can try different hyperparameters, starting with simple models first.  The choice of loss function can also be a hyperparameter, but you can try common loss functions like crossentropy loss (CEL) and root mean squared error (RMSE) first.
  
  The output units can have a sigmoid activation so that all the values are between $0$ and $1$.  I'm not sure how well a unit step function would work here as an activation function, but that would guarantee that the values are either $0$ or $1$.  With the unit step function, there is no need for post-processing.  You can experiment with both.
  
  If you use a sigmoid activation, then you will still need to do some post-processing to ensure that the final values in the output vector are $0$ or $1$. For example, you can set values $>0.95 = 1$ and all others $= 0$.
  
  
  ---
  
  