title: Modeling Encoder-Decoder according to instructions from a paper
body: <p>I am new to this field and I was reading a paper &quot;Predicting citation counts based on deep neural network learning techniques&quot;. There the authors describe the code that they implemented if someone wants to reproduce the results. I tried to do this but I am not sure if I succeeded.</p>
  <p>Here is their description:</p>
  <pre><code>-RNN module - SimpleRNN
  -Output dimension of the encoder - 512
  -The output layer - Dense layer
  -Activation function - ReLU
  -Overfitting prevention technique - Dropout with 0.2 rate
  -Epochs - 100
  Optimization algorithm - RMSProp
  Learning rate - 10^{-5}
  Batch size - 256
  </code></pre>
  <p>And here is my implementation. I am not sure if the model I created is sequence to sequence.</p>
  <pre><code>epocsh = 100
  batch_size = 256
  optimizer = keras.optimizers.RMSprop(lr=0.00001)
  model =  keras.models.Sequential([
      keras.layers.SimpleRNN(512, input_shape=[X_train.shape[0], X_train.shape[1]],
                             activation='relu', return_sequences=True, dropout=0.2),
      keras.layers.Dense(9)
  ])
  
  model.compile(loss='mse', optimizer=optimizer, metrics=[keras.metrics.RootMeanSquaredError()])
  </code></pre>
  <p>The summary of this model is:</p>
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  simple_rnn (SimpleRNN)       (None, 154521, 512)       266240    
  _________________________________________________________________
  dense (Dense)                (None, 154521, 9)         4617      
  =================================================================
  Total params: 270,857
  Trainable params: 270,857
  Non-trainable params: 0
  _________________________________________________________________
  </code></pre>
  <p>Update: Is this maybe the correct way to formulate this?</p>
  <pre><code>encoder = keras.layers.SimpleRNN(512,
                                   input_shape=[X_train.shape[0], X_train.shape[1]],
                                   activation='relu',
                                   return_sequences=False,
                                   dropout=0.2)
  
  decoder = keras.layers.SimpleRNN(512,
                                   input_shape=[X_train.shape[0], X_train.shape[1]],
                                   activation='relu',
                                   return_sequences=True,
                                   dropout=0.2)
  
  output = keras.layers.Dense(9)(decoder)
  </code></pre>
  <p>This is the dataset that I am using.</p>
  <pre><code>year  venue  c1  c2  c3  c4  c5  c6  c7  c8  c9  c10  c11  c12  c13  c14
  1989    234   0   1   2   3   4   5   5   5   5    8    8   10   11   12
  1989    251   0   0   0   0   0   0   0   0   0    0    0    0    0    0
  1990    346   0   0   0   0   0   0   0   0   0    0    0    0    0    0
  </code></pre>
  <p>I need to give as an input all the columns until c5, and try to predict the other c's (which are citation count for the upcoming years). Is this the right way to go forward?</p>