Answers.Body: <p>To get a full understanding of your problem, one would like to know what approximately the <span class="math-container">$n$</span>-features are.</p>
  <p>Whether, it is about the geometrical structure, protein is described by a graph, where vertices correspond to atoms and edges to bonds within them - I would consider use of <code>GraphNN</code>, there is some research, that has demonstrated the success of <code>GraphNN</code> for protein prediction:</p>
  <ul>
  <li><a href="https://www.nature.com/articles/s41467-021-23303-9" rel="nofollow noreferrer">Structure-based protein function prediction using graph convolutional networks</a></li>
  <li><a href="https://proceedings.neurips.cc/paper/2017/file/f507783927f2ec2737ba40afbd17efb5-Paper.pdf" rel="nofollow noreferrer">Protein Interface Prediction using Graph Convolutional Networks</a></li>
  </ul>
  <p>In case, your data is some general form tabular data with features of different nature and form - like some <strong>continuous</strong> features, <strong>binary</strong> features, <strong>categorical</strong> features, whatever, there is no notion of proximity between different features.</p>
  <p>The efficiency of CNN is based a lot on the ability of them to have a notion of locality, aggregate the information from the neighborhood, and construct the hierarchy of low-level (for several neighboring pixels) features and global (that understand the image or a signal as a whole) features.</p>
  <p>Two neighboring pixes from the cat's ear have a notion of proximity and relatedness, whereas the red color and square shape do not have.</p>
  <p>For unstructured tabular data I would recommend to start from some <code>tree ensembling approach</code>:</p>
  <ul>
  <li>Random forest</li>
  <li>Gradient boosting</li>
  </ul>
  <p><strong>EDIT</strong></p>
  <p>Seems like what you receive as input is a matrix of pairwise distances <span class="math-container">$\rho(i, j)$</span> between all possible amino residues - <a href="https://en.wikipedia.org/wiki/Protein_contact_map" rel="nofollow noreferrer">Protein contact map</a>.</p>
  <p>I would think about it as a weighted adjacency matrix. However, there is a rather special structure, <a href="https://i.sstatic.net/Rk3fK.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Rk3fK.jpg" alt="enter image description here" /></a></p>
  <p>Therefore, there is no need for generic GNN, most likely.</p>
  <p>In the literature, the recent research solves this problem with the help of CNN:</p>
  <ul>
  <li><a href="https://www.pnas.org/content/117/3/1496" rel="nofollow noreferrer">Improved protein structure prediction using predicted interresidue orientations</a></li>
  <li><a href="https://www.nature.com/articles/s41598-020-70181-0" rel="nofollow noreferrer">A fully open-source framework for deep learning protein real-valued distances</a></li>
  </ul>
  <p>Probably, you should tune some hyperparameters, or introduce residual connections, if there are no such at the moment.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-27T23:34:20.607Z
Answers.Text: What do you mean by residual connection?
Answers.UserDisplayName: user366312
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-28T04:13:59.693Z
Answers.Text: @user366312 for deep neural networks - with the number of layers more then 10, say, networks become very hard to optimize and accuracy saturates at rather high loss (train and test). I was a breakthrough, when [Resnets](https://arxiv.org/pdf/1512.03385.pdf) were introduced, where after each layer, without downsampling or change of number of channels, one adds the input of the stack of layers to the output. [Concatenation](https://paperswithcode.com/method/densenet) is possible as well. This combats the problems with optimization.
Answers.UserDisplayName: spiridon_the_sun_rotator
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-21T07:12:18.490Z
Answers.LastActivityDate: 2021-09-22T04:35:49.017Z
Answers.LastEditDate: 2021-09-22T04:35:49.017Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Body: <p>I fed the same set of 1.4 million data to two different models:</p>
  <ol>
  <li>MLP</li>
  <li>CNN model</li>
  </ol>
  <p>In both cases, I used the same parameters and hyperparameters.</p>
  <p>The CNN is showing comparatively lower accuracy (80%) than that of the MLP (82%).</p>
  <p>Why?</p>
  <p>And, also, what does this experiment tell us?</p>
  <p><strong>Edit:</strong></p>
  <blockquote>
  <p>Is the data images, video or audio, or other grid-based signals (same signal repeated at multiple locations with a meaningful distance metric between them) in your case?</p>
  </blockquote>
  <p>It is <strong>protein</strong> C-alpha distances data that has 3 classes (helix, strand, coil) and <em>n</em>-number of features, where <em>n</em> is an even number.</p>
  <blockquote>
  <p>In fact, what is it, what was your expectation of the relative performance of the two models, and why?</p>
  </blockquote>
  <p>I thought CNN would be more efficient and thereby would demonstrate better test/validation accuracy.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-21T06:20:43.830Z
Comments.Text: Is the data images, video or audio, or other grid-based signal (same signal repeated at multiple locations with a meaningful distance metric between them) in your case? In fact what is it, what was your expectation of relative performance of the two models, and why?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-21T06:50:39.413Z
Comments.Text: @NeilSlater, see the edit.
Comments.UserDisplayName: user366312
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-21T07:03:22.487Z
Comments.Text: Thanks for the edit. Sorry, I still don't understand your description of the data. A protein could be represented by a 1d signal of identical features (e.g. an array of amino acids), but could be anything else. I need more. E.g. If I said my data was "car data" you would have no clue whether it was pictures of cars, measurements of dimensions, engine stats, accident stats etc. Please give something more useful to work with. Also, while I am asking, how have you "balanced" between the models in terms of number of parameters (perhaps give the basic number of neurons etc).
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-21T07:23:59.047Z
Comments.Text: @NeilSlater, C-alpha distances.
Comments.UserDisplayName: user366312
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-21T09:02:18.263Z
Comments.Text: So you are doing something like this: https://www.researchgate.net/publication/253393184_Using_Distances_Between_alpha-Carbons_to_Predict_Protein_Structure ?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-21T10:22:35.177Z
Comments.Text: @NeilSlater, Yes...
Comments.UserDisplayName: user366312
Tags: <neural-networks><convolutional-neural-networks><models>
Title: Why is the validation accuracy lower in case of CNN?
UserDisplayName: user366312
ThreadText: # Why is the validation accuracy lower in case of CNN?
  
  (Asked by: username_0 on 2021-09-21)
  
  I fed the same set of 1.4 million data to two different models:
  
  1. MLP
  2. CNN model
  
  In both cases, I used the same parameters and hyperparameters.
  
  The CNN is showing comparatively lower accuracy (80%) than that of the MLP (82%).
  
  Why?
  
  And, also, what does this experiment tell us?
  
  **Edit:**
  
  > Is the data images, video or audio, or other grid-based signals (same signal repeated at multiple locations with a meaningful distance metric between them) in your case?
  
  It is **protein** C-alpha distances data that has 3 classes (helix, strand, coil) and *n*-number of features, where *n* is an even number.
  
  > In fact, what is it, what was your expectation of the relative performance of the two models, and why?
  
  I thought CNN would be more efficient and thereby would demonstrate better test/validation accuracy.
  
  **Tags:** neural-networks, convolutional-neural-networks, models
  > **Comments:**
  > **username_1**: Is the data images, video or audio, or other grid-based signal (same signal repeated at multiple locations with a meaningful distance metric between them) in your case? In fact what is it, what was your expectation of relative performance of the two models, and why?  
  > **username_0**: @NeilSlater, see the edit.  
  > **username_1**: Thanks for the edit. Sorry, I still don't understand your description of the data. A protein could be represented by a 1d signal of identical features (e.g. an array of amino acids), but could be anything else. I need more. E.g. If I said my data was "car data" you would have no clue whether it was pictures of cars, measurements of dimensions, engine stats, accident stats etc. Please give something more useful to work with. Also, while I am asking, how have you "balanced" between the models in terms of number of parameters (perhaps give the basic number of neurons etc).  
  > **username_0**: @NeilSlater, C-alpha distances.  
  > **username_1**: So you are doing something like this: https://www.researchgate.net/publication/253393184_Using_Distances_Between_alpha-Carbons_to_Predict_Protein_Structure ?  
  > **username_0**: @NeilSlater, Yes...  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-09-21. Score: 2
  
  To get a full understanding of your problem, one would like to know what approximately the $n$-features are.
  
  Whether, it is about the geometrical structure, protein is described by a graph, where vertices correspond to atoms and edges to bonds within them - I would consider use of `GraphNN`, there is some research, that has demonstrated the success of `GraphNN` for protein prediction:
  
  * [Structure-based protein function prediction using graph convolutional networks](https://www.nature.com/articles/s41467-021-23303-9)
  * [Protein Interface Prediction using Graph Convolutional Networks](https://proceedings.neurips.cc/paper/2017/file/f507783927f2ec2737ba40afbd17efb5-Paper.pdf)
  
  In case, your data is some general form tabular data with features of different nature and form - like some **continuous** features, **binary** features, **categorical** features, whatever, there is no notion of proximity between different features.
  
  The efficiency of CNN is based a lot on the ability of them to have a notion of locality, aggregate the information from the neighborhood, and construct the hierarchy of low-level (for several neighboring pixels) features and global (that understand the image or a signal as a whole) features.
  
  Two neighboring pixes from the cat's ear have a notion of proximity and relatedness, whereas the red color and square shape do not have.
  
  For unstructured tabular data I would recommend to start from some `tree ensembling approach`:
  
  * Random forest
  * Gradient boosting
  
  **EDIT**
  
  Seems like what you receive as input is a matrix of pairwise distances $\rho(i, j)$ between all possible amino residues - [Protein contact map](https://en.wikipedia.org/wiki/Protein_contact_map).
  
  I would think about it as a weighted adjacency matrix. However, there is a rather special structure, [![enter image description here](https://i.sstatic.net/Rk3fK.jpg)](https://i.sstatic.net/Rk3fK.jpg)
  
  Therefore, there is no need for generic GNN, most likely.
  
  In the literature, the recent research solves this problem with the help of CNN:
  
  * [Improved protein structure prediction using predicted interresidue orientations](https://www.pnas.org/content/117/3/1496)
  * [A fully open-source framework for deep learning protein real-valued distances](https://www.nature.com/articles/s41598-020-70181-0)
  
  Probably, you should tune some hyperparameters, or introduce residual connections, if there are no such at the moment.
  
  > **Comments:**
  > **username_0**: What do you mean by residual connection?  
  > **username_2**: @user366312 for deep neural networks - with the number of layers more then 10, say, networks become very hard to optimize and accuracy saturates at rather high loss (train and test). I was a breakthrough, when [Resnets](https://arxiv.org/pdf/1512.03385.pdf) were introduced, where after each layer, without downsampling or change of number of channels, one adds the input of the stack of layers to the output. [Concatenation](https://paperswithcode.com/method/densenet) is possible as well. This combats the problems with optimization.  
  
  ---
  
  