Body: <p>I am working with a dataset where each input sample is a matrix, and the output corresponding to each input is also a matrix (of shape <code>(400, 10)</code>). The <strong>input samples do not have translation invariance</strong>. Each output image has shape <code>(16, 16)</code> . The output matrices have translation invariance. </p>
  
  <p><strong>I want to build a neural network which can learn how to predict the output images</strong> from the aforementioned data. It seems to me that one needs to think of a <strong>neural network here which does regression on the output images</strong> to learn. Presently, I am using <strong>1000 data samples for learning in the neural network</strong> (input samples and corresponding output images). <strong>What is the best way to build a neural network for this task?</strong></p>
  
  <p>Presently, I am using multi-layer perceptron (MLP) with mean square error (MSE) loss for this task. I flatten the input matrices before I feed them into the MLP, and use a standard MLP with multiple hidden layers (4-5) with many hidden units for this task. </p>
  
  <p>While a visual inspection shows that the true and predicted output images are in relatively good agreement for training data, I find a <strong>mismatch between true and predicted (reconstructed) output images for validation data</strong>. The bottom plot shows pictures of true and predicted (reconstructed) output images for training and validation data for chosen samples. </p>
  
  <p><a href="https://i.sstatic.net/Ns9FP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Ns9FP.png" alt="enter image description here"></a></p>
  
  <p>Presently, I am using training and validation loss curves (with respect to iteration) to measure performance. I want to have a robust metric for comparison which can tell me whether the prediction is a random image or not.</p>
  
  <p><strong>How can I get the model to generalize to the validation set better?</strong></p>
  
  <p>The Python code that I am using for this MLP and the required data can be found <a href="https://drive.google.com/file/d/1_Ls2Q6m-Df96JjhumHchy25kpddfCZSt/view?usp=sharing" rel="nofollow noreferrer">here</a> and <a href="https://drive.google.com/file/d/1YaHmP-amz14Ev4nh5rsPi4l7XDDGq3RL/view?usp=sharing" rel="nofollow noreferrer">here</a> respectively.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-04T06:02:29.577Z
Comments.Text: Hi. Welcome to AI.SE. Your post is more likely to get responded to if it consists of a single question. Also, implementation questions are off-topic here. This post has the best chance of not being closed as off-topic if you focus on conceptual questions. For more information, [see these](https://ai.stackexchange.com/help/on-topic) [help pages](https://ai.stackexchange.com/help/how-to-ask).
Comments.UserDisplayName: Philip Raeisghasem
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-04T06:08:47.387Z
Comments.Text: Hi Philip, thanks for your suggestion. I have edited the question after your suggestions. I hope it is fine now :)
Comments.UserDisplayName: Siddharth Satpathy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-04T10:18:42.767Z
Comments.Text: By saying that the input samples do not have translation invariance, do you mean that they are not images? You also say "the output corresponding to each input is also a matrix (of shape `(400, 10)`)", but then you say "Each output image has shape `(16, 16)`", so what is the difference between the output corresponding to each input sample and the output image (given that they have different dimensions)? I think you should edit your question to clarify this point (at least it is not clear to me).
Comments.UserDisplayName: nbro
Tags: <neural-networks><machine-learning><deep-learning>
Title: How to build a neural network that can learn to predict output images?
UserDisplayName: Siddharth Satpathy
ThreadText: # How to build a neural network that can learn to predict output images?
  
  (Asked by: username_0 on 2019-04-04)
  
  I am working with a dataset where each input sample is a matrix, and the output corresponding to each input is also a matrix (of shape `(400, 10)`). The **input samples do not have translation invariance**. Each output image has shape `(16, 16)` . The output matrices have translation invariance.
  
  **I want to build a neural network which can learn how to predict the output images** from the aforementioned data. It seems to me that one needs to think of a **neural network here which does regression on the output images** to learn. Presently, I am using **1000 data samples for learning in the neural network** (input samples and corresponding output images). **What is the best way to build a neural network for this task?**
  
  Presently, I am using multi-layer perceptron (MLP) with mean square error (MSE) loss for this task. I flatten the input matrices before I feed them into the MLP, and use a standard MLP with multiple hidden layers (4-5) with many hidden units for this task.
  
  While a visual inspection shows that the true and predicted output images are in relatively good agreement for training data, I find a **mismatch between true and predicted (reconstructed) output images for validation data**. The bottom plot shows pictures of true and predicted (reconstructed) output images for training and validation data for chosen samples.
  
  [![enter image description here](https://i.sstatic.net/Ns9FP.png)](https://i.sstatic.net/Ns9FP.png)
  
  Presently, I am using training and validation loss curves (with respect to iteration) to measure performance. I want to have a robust metric for comparison which can tell me whether the prediction is a random image or not.
  
  **How can I get the model to generalize to the validation set better?**
  
  The Python code that I am using for this MLP and the required data can be found [here](https://drive.google.com/file/d/1_Ls2Q6m-Df96JjhumHchy25kpddfCZSt/view?usp=sharing) and [here](https://drive.google.com/file/d/1YaHmP-amz14Ev4nh5rsPi4l7XDDGq3RL/view?usp=sharing) respectively.
  
  **Tags:** neural-networks, machine-learning, deep-learning
  > **Comments:**
  > **username_1**: Hi. Welcome to AI.SE. Your post is more likely to get responded to if it consists of a single question. Also, implementation questions are off-topic here. This post has the best chance of not being closed as off-topic if you focus on conceptual questions. For more information, [see these](https://ai.stackexchange.com/help/on-topic) [help pages](https://ai.stackexchange.com/help/how-to-ask).  
  > **username_0**: Hi Philip, thanks for your suggestion. I have edited the question after your suggestions. I hope it is fine now :)  
  > **username_2**: By saying that the input samples do not have translation invariance, do you mean that they are not images? You also say "the output corresponding to each input is also a matrix (of shape `(400, 10)`)", but then you say "Each output image has shape `(16, 16)`", so what is the difference between the output corresponding to each input sample and the output image (given that they have different dimensions)? I think you should edit your question to clarify this point (at least it is not clear to me).  
  
  ---
  
  ## 0 answers
  
  