Answers.Body: <p>The link you have mentioned is using Dense layers. One thing to start with would be to use 1D CNNs (they will capture some local information). Also, since sequence matters in your case, refrain from one-hot encodings (just 1, 2, 3, 4). And for the 2D matrix, use a 2D CNN. Then, flatten your encoding for both 1D CNN and 2D CNN, then, finally combine them. This shall give you some improvement.</p>
  <p>This is for your network your are already using.</p>
  <p>Here is an alternate suggestion for you:</p>
  <p>Look at GCN (Graph Convolutional Network). Since, you have a graph structure (60x60 matrix is actually an adjacency matrix) and each node i.e. 60x4 in your case, can then become node features. This is what you want a GNN and its variants  for. Look at GAT (Graph Attention Networks) if GCN don't work for you.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-16T07:56:22.403Z
Answers.Text: Dear Abhishek, thank you for your response! I actually have tried 1D CNN for the sequence and 2D CNN. I want to add a convolutional layer after I combine them, but I am having trouble. Do you know how to do this? I have added the relevant code to my original question
Answers.UserDisplayName: gollyzoom
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-16T21:08:49.757Z
Answers.Text: Put `combined_input` through a Dense layer. Also, return Flattened tensors (See Flatten Layer) from both the functions.
Answers.UserDisplayName: Abhishek Verma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-10T23:06:36.287Z
Answers.LastActivityDate: 2021-02-10T23:06:36.287Z
Answers.UserDisplayName: Abhishek Verma
Body: <p>I am relatively new to machine learning, and I am trying to use a deep neural network to extract some information from sequences of RNA.</p>
  <p>A quick overview of RNA: there is both sequence and structure. I am currently expressing the sequence with one-hot encoding (so a sequence of length <span class="math-container">$60$</span> would be expressed as a <span class="math-container">$60 \times 4$</span> matrix, with one row for each letter of the sequence, and one column for each possible value of that letter). I am also feeding the 2D structure of the RNA into the network, which is expressed as <span class="math-container">$60 \times 60$</span> matrix for a sequence of length <span class="math-container">$60$</span>.</p>
  <p>I am trying to use these inputs to predict a single continuous value for a given sequence.</p>
  <p>Currently, I am using pretty much the exact setup <a href="https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/" rel="nofollow noreferrer">from this tutorial</a>. I chose this architecture because it allows me to separate the inputs (sequence and structure) and have individual layers for them before merging them into a single model. I think this makes more sense than trying to glue the two separate pieces of data together into a single input.</p>
  <p>However, the model doesn't seem to be learning anything - validation loss decreases very slightly then plateaus.</p>
  <p>If anyone has suggestions, especially someone who has worked with RNA, DNA, or proteins before, I would really, really appreciate it. I am new to this, and I am not sure how to improve my model from here.</p>
  <pre><code>def create_mlp(height,width,filters=(16, 16, 32, 32, 64), regress=False):
      # initialize the input shape and channel dimension, assuming
    # TensorFlow/channels-last ordering
    inputShape = (height, width)
    chanDim = -1
    # define the model input
    inputs = Input(shape=inputShape)
    # loop over the number of filters
    for (i, f) in enumerate(filters):
      # if this is the first CONV layer then set the input
      # appropriately
      if i == 0:
        x = inputs
      # CONV =&gt; RELU =&gt; BN =&gt; POOL
      x = Conv1D(f, 3, padding=&quot;same&quot;)(x)
      x = Activation(&quot;relu&quot;)(x)
      x = BatchNormalization(axis=chanDim)(x)
      x = MaxPooling1D(pool_size=2)(x)
    # flatten the volume, then FC =&gt; RELU =&gt; BN =&gt; DROPOUT
    print(x.shape)
    x = Flatten()(x)
    x = Dense(16)(x)
    x = Activation(&quot;relu&quot;)(x)
    x = BatchNormalization(axis=chanDim)(x)
    x = Dropout(0.5)(x)
    # apply another FC layer, this one to match the number of nodes
    # coming out of the MLP
    x = Dense(4)(x)
    x = Activation(&quot;relu&quot;)(x)
    # check to see if the regression node should be added
    if regress:
      x = Dense(1, activation=&quot;linear&quot;)(x)
    # construct the CNN
    model = Model(inputs, x)
    # return the CNN
    return model
  
  def create_cnn(width, height, depth, filters=(16, 16, 32, 32, 64), regress=False):
    # initialize the input shape and channel dimension, assuming
    # TensorFlow/channels-last ordering
    inputShape = (height, width, depth)
    chanDim = -1
    # define the model input
    inputs = Input(shape=inputShape)
    # loop over the number of filters
    for (i, f) in enumerate(filters):
      # if this is the first CONV layer then set the input
      # appropriately
      if i == 0:
        x = inputs
      # CONV =&gt; RELU =&gt; BN =&gt; POOL
      x = Conv2D(f, (3, 3), padding=&quot;same&quot;)(x)
      x = Activation(&quot;relu&quot;)(x)
      x = BatchNormalization(axis=chanDim)(x)
      x = MaxPooling2D(pool_size=(2, 2))(x)
    # flatten the volume, then FC =&gt; RELU =&gt; BN =&gt; DROPOUT
    print(x.shape)
    x = Flatten()(x)
    x = Dense(16)(x)
    x = Activation(&quot;relu&quot;)(x)
    x = BatchNormalization(axis=chanDim)(x)
    x = Dropout(0.5)(x)
    # apply another FC layer, this one to match the number of nodes
    # coming out of the MLP
    x = Dense(4)(x)
    x = Activation(&quot;relu&quot;)(x)
    # check to see if the regression node should be added
    if regress:
      x = Dense(1, activation=&quot;linear&quot;)(x)
    # construct the CNN
    model = Model(inputs, x)
    # return the CNN
    return model
  
  mlp = create_mlp(l, 4, regress=False)
    cnn = create_cnn(l, l, 1, regress=False)
    # create the input to our final set of layers as the *output* of both
    # the MLP and CNN
    #cnn.output.reshape()
    combinedInput = concatenate([mlp.output, cnn.output])
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-10T10:21:21.023Z
Comments.Text: Hello. Welcome to [Artificial Intelligence Stack Exchange](https://ai.stackexchange.com/help). Can you please clarify what you're trying to predict for each sequence/RNA? Moreover, can you clarify why your network is a $60 \times 60$ matrix? This means that if you multiply your input matrix $60 \times 4$ from the right with your first weight matrix you will get another $60 \times 4$ matrix. If this is correct, what do you do then with this new $60 \times 4$ matrix? To get a final number (which is what you're doing, given that you're doing regression), you need to parse this matrix, but how?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-10T10:23:21.283Z
Comments.Text: Additionally, I would also suggest that you show us the plot of the validation loss, specify how many epochs you trained your model for, what is the size of your training and validation dataset, learning rate, number of parameters of the net, and any other metric you're computing to assess the quality of the predictions/regressions. Finally, please, try to ask a specific question and not just "any suggestions to solve this problem are welcome", then put it in the title so that people immediately understand what you're asking here.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-10T10:26:45.217Z
Comments.Text: Sorry if these are too many questions at the same time, but sometimes, even though we think that our post/question is clear to us, it's not enough to clear to other people (at least all the possible details that may be helpful to provide an answer may not be available).
Comments.UserDisplayName: nbro
Tags: <neural-networks><tensorflow><prediction><biology>
Title: Extracting information from RNA sequence
UserDisplayName: gollyzoom
ThreadText: # Extracting information from RNA sequence
  
  (Asked by: username_0 on 2021-02-10)
  
  I am relatively new to machine learning, and I am trying to use a deep neural network to extract some information from sequences of RNA.
  A quick overview of RNA: there is both sequence and structure. I am currently expressing the sequence with one-hot encoding (so a sequence of length $60$ would be expressed as a $60 \times 4$ matrix, with one row for each letter of the sequence, and one column for each possible value of that letter). I am also feeding the 2D structure of the RNA into the network, which is expressed as $60 \times 60$ matrix for a sequence of length $60$.
  I am trying to use these inputs to predict a single continuous value for a given sequence.
  Currently, I am using pretty much the exact setup from this tutorial. I chose this architecture because it allows me to separate the inputs (sequence and structure) and have individual layers for them before merging them into a single model. I think this makes more sense than trying to glue the two separate pieces of data together into a single input.
  However, the model doesn't seem to be learning anything - validation loss decreases very slightly then plateaus.
  If anyone has suggestions, especially someone who has worked with RNA, DNA, or proteins before, I would really, really appreciate it. I am new to this, and I am not sure how to improve my model from here.
  def create_mlp(height,width,filters=(16, 16, 32, 32, 64), regress=False):
      # initialize the input shape and channel dimension, assuming
    # TensorFlow/channels-last ordering
    inputShape = (height, width)
    chanDim = -1
    # define the model input
    inputs = Input(shape=inputShape)
    # loop over the number of filters
    for (i, f) in enumerate(filters):
      # if this is the first CONV layer then set the input
      # appropriately
      if i == 0:
        x = inputs
      # CONV => RELU => BN => POOL
      x = Conv1D(f, 3, padding="same")(x)
      x = Activation("relu")(x)
      x = BatchNormalization(axis=chanDim)(x)
      x = MaxPooling1D(pool_size=2)(x)
    # flatten the volume, then FC => RELU => BN => DROPOUT
    print(x.shape)
    x = Flatten()(x)
    x = Dense(16)(x)
    x = Activation("relu")(x)
    x = BatchNormalization(axis=chanDim)(x)
    x = Dropout(0.5)(x)
    # apply another FC layer, this one to match the number of nodes
    # coming out of the MLP
    x = Dense(4)(x)
    x = Activation("relu")(x)
    # check to see if the regression node should be added
    if regress:
      x = Dense(1, activation="linear")(x)
    # construct the CNN
    model = Model(inputs, x)
    # return the CNN
    return model
  
  def create_cnn(width, height, depth, filters=(16, 16, 32, 32, 64), regress=False):
    # initialize the input shape and channel dimension, assuming
    # TensorFlow/channels-last ordering
    inputShape = (height, width, depth)
    chanDim = -1
    # define the model input
    inputs = Input(shape=inputShape)
    # loop over the number of filters
    for (i, f) in enumerate(filters):
      # if this is the first CONV layer then set the input
      # appropriately
      if i == 0:
        x = inputs
      # CONV => RELU => BN => POOL
      x = Conv2D(f, (3, 3), padding="same")(x)
      x = Activation("relu")(x)
      x = BatchNormalization(axis=chanDim)(x)
      x = MaxPooling2D(pool_size=(2, 2))(x)
    # flatten the volume, then FC => RELU => BN => DROPOUT
    print(x.shape)
    x = Flatten()(x)
    x = Dense(16)(x)
    x = Activation("relu")(x)
    x = BatchNormalization(axis=chanDim)(x)
    x = Dropout(0.5)(x)
    # apply another FC layer, this one to match the number of nodes
    # coming out of the MLP
    x = Dense(4)(x)
    x = Activation("relu")(x)
    # check to see if the regression node should be added
    if regress:
      x = Dense(1, activation="linear")(x)
    # construct the CNN
    model = Model(inputs, x)
    # return the CNN
    return model
  
  mlp = create_mlp(l, 4, regress=False)
    cnn = create_cnn(l, l, 1, regress=False)
    # create the input to our final set of layers as the *output* of both
    # the MLP and CNN
    #cnn.output.reshape()
    combinedInput = concatenate([mlp.output, cnn.output])
  
  **Tags:** neural-networks, tensorflow, prediction, biology
  > **Comments:**
  > **username_1**: Hello. Welcome to [Artificial Intelligence Stack Exchange](https://ai.stackexchange.com/help). Can you please clarify what you're trying to predict for each sequence/RNA? Moreover, can you clarify why your network is a $60 \times 60$ matrix? This means that if you multiply your input matrix $60 \times 4$ from the right with your first weight matrix you will get another $60 \times 4$ matrix. If this is correct, what do you do then with this new $60 \times 4$ matrix? To get a final number (which is what you're doing, given that you're doing regression), you need to parse this matrix, but how?  
  > **username_1**: Additionally, I would also suggest that you show us the plot of the validation loss, specify how many epochs you trained your model for, what is the size of your training and validation dataset, learning rate, number of parameters of the net, and any other metric you're computing to assess the quality of the predictions/regressions. Finally, please, try to ask a specific question and not just "any suggestions to solve this problem are welcome", then put it in the title so that people immediately understand what you're asking here.  
  > **username_1**: Sorry if these are too many questions at the same time, but sometimes, even though we think that our post/question is clear to us, it's not enough to clear to other people (at least all the possible details that may be helpful to provide an answer may not be available).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-02-10. Score: 1
  
  The link you have mentioned is using Dense layers. One thing to start with would be to use 1D CNNs (they will capture some local information). Also, since sequence matters in your case, refrain from one-hot encodings (just 1, 2, 3, 4). And for the 2D matrix, use a 2D CNN. Then, flatten your encoding for both 1D CNN and 2D CNN, then, finally combine them. This shall give you some improvement.
  
  This is for your network your are already using.
  
  Here is an alternate suggestion for you:
  
  Look at GCN (Graph Convolutional Network). Since, you have a graph structure (60x60 matrix is actually an adjacency matrix) and each node i.e. 60x4 in your case, can then become node features. This is what you want a GNN and its variants for. Look at GAT (Graph Attention Networks) if GCN don't work for you.
  
  > **Comments:**
  > **username_0**: Dear Abhishek, thank you for your response! I actually have tried 1D CNN for the sequence and 2D CNN. I want to add a convolutional layer after I combine them, but I am having trouble. Do you know how to do this? I have added the relevant code to my original question  
  > **username_2**: Put `combined_input` through a Dense layer. Also, return Flattened tensors (See Flatten Layer) from both the functions.  
  
  ---
  
  