Answers.Body: <p>A time series, usually, requires regular time intervals, but, from looking at your example, it seems that's not the case. You could try to use a MLP and give it as input the Time and Bitrate pairs and make it output the Class.</p>
  <p>The activation function is what makes your neural network produce its output, i.e. activate the neurons. The loss function calculates the error of your model's predictions. This error is used by the backpropagation algorithm to adjust the model when training.</p>
  <p>If your data can only be in one of two classes, for example 0 or 1, you have a binary classification problem. I recommend using the <a href="https://en.wikipedia.org/wiki/Sigmoid_function" rel="nofollow noreferrer">Sigmoid</a> as the activation function. If you look at the Sigmoid's plot, you'll notice that, no matter the input, the output will always be within 0 and 1. You can think of this as the probability of the input being in class 0 or 1. So, for example, any input that produces a probability under 0.5 is class 0. Of course, if you want to be more precise you can simply procure the probabilities without the making it fit the labels. For example, an output of 0.7 represents that there is a 70% probability of the input belonging to class 1. Another term for this is <a href="https://en.wikipedia.org/wiki/Logistic_regression" rel="nofollow noreferrer">Logistic Regression</a>.</p>
  <p>The <a href="https://towardsdatascience.com/understanding-binary-cross-entropy-log-loss-a-visual-explanation-a3ac6025181a" rel="nofollow noreferrer">Binary cross-entropy</a> calculates the error between the model's prediction and the real class label. This loss function can be applied to many classes, but the binary version keeps it to 0 and 1.</p>
  <p>Choosing the right activation functions and loss functions will depend on the problem you are trying to solve. Here are some useful guides  on <a href="https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/" rel="nofollow noreferrer">loss functions</a> and <a href="https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/" rel="nofollow noreferrer">activation functions</a>.</p>
  <p>Additional to  the above, try to use <a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html" rel="nofollow noreferrer">scaling(normalize)</a> the inputs before feeding to the network, so that the network may generalize, as it looks like bitrate value above 300 is likely to be associated with class 1. It may improve performance.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-29T16:10:00.183Z
Answers.Text: Thanks Marcus! Do you have any useful tutorial etc. advice to understand what kind of loss functions should be used with which activations or vice-versa?
Answers.UserDisplayName: bbasaran
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-29T20:05:37.870Z
Answers.Text: @bbasaran I updated the answer. If you are interested in a deep learning course, I would recommend Coursera's Deep Learning course.
Answers.UserDisplayName: Marcus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-28T20:07:37.523Z
Answers.LastActivityDate: 2021-05-08T17:03:05.480Z
Answers.LastEditDate: 2021-05-08T17:03:05.480Z
Answers.UserDisplayName: Marcus
Body: <p>Let's say I have the time-series dataset below-left. I would like to train a model in such a way that, if I feed the model with an input like the test sequence below, it should be able to classify each sample with the correct class label.</p>
  <pre><code>    Training Sequence:              Test Sequence:
  
  Time,   Bitrate,   Class           Time,   Bitrate    Predicted Class
   0,       312,       1              0,       234      -----&gt; 0
   0.3,     319,       1              0.2,     261      -----&gt; 0
   0.5,     227,       0              0.4,     277      -----&gt; 0
   0.6,     229,       0              0.7,     301      -----&gt; 1 
   0.7,     219,       0              0.8,     305      -----&gt; 1
   0.8,     341,       1              0.9,     343      -----&gt; 1 
   0.9,     281,       0              1.0,     299      -----&gt; 0 
            ...                                ...
  </code></pre>
  <p>So, what kind of neural network should I build to classify each instance of a time series sequence?</p>
  
Tags: <classification><long-short-term-memory><time-series><algorithm-request><model-request>
Title: What kind of neural network should I build to classify each instance of a time series sequence?
UserDisplayName: bbasaran
ThreadText: # What kind of neural network should I build to classify each instance of a time series sequence?
  
  (Asked by: username_0 on 2021-04-28)
  
  Let's say I have the time-series dataset below-left. I would like to train a model in such a way that, if I feed the model with an input like the test sequence below, it should be able to classify each sample with the correct class label.
      Training Sequence:              Test Sequence:
  
  Time,   Bitrate,   Class           Time,   Bitrate    Predicted Class
   0,       312,       1              0,       234      -----> 0
   0.3,     319,       1              0.2,     261      -----> 0
   0.5,     227,       0              0.4,     277      -----> 0
   0.6,     229,       0              0.7,     301      -----> 1 
   0.7,     219,       0              0.8,     305      -----> 1
   0.8,     341,       1              0.9,     343      -----> 1 
   0.9,     281,       0              1.0,     299      -----> 0 
            ...                                ...
  
  So, what kind of neural network should I build to classify each instance of a time series sequence?
  
  **Tags:** classification, long-short-term-memory, time-series, algorithm-request, model-request
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-04-28. Score: 1 | Accepted answer
  
  A time series, usually, requires regular time intervals, but, from looking at your example, it seems that's not the case. You could try to use a MLP and give it as input the Time and Bitrate pairs and make it output the Class.
  
  The activation function is what makes your neural network produce its output, i.e. activate the neurons. The loss function calculates the error of your model's predictions. This error is used by the backpropagation algorithm to adjust the model when training.
  
  If your data can only be in one of two classes, for example 0 or 1, you have a binary classification problem. I recommend using the [Sigmoid](https://en.wikipedia.org/wiki/Sigmoid_function) as the activation function. If you look at the Sigmoid's plot, you'll notice that, no matter the input, the output will always be within 0 and 1. You can think of this as the probability of the input being in class 0 or 1. So, for example, any input that produces a probability under 0.5 is class 0. Of course, if you want to be more precise you can simply procure the probabilities without the making it fit the labels. For example, an output of 0.7 represents that there is a 70% probability of the input belonging to class 1. Another term for this is [Logistic Regression](https://en.wikipedia.org/wiki/Logistic_regression).
  
  The [Binary cross-entropy](https://towardsdatascience.com/understanding-binary-cross-entropy-log-loss-a-visual-explanation-a3ac6025181a) calculates the error between the model's prediction and the real class label. This loss function can be applied to many classes, but the binary version keeps it to 0 and 1.
  
  Choosing the right activation functions and loss functions will depend on the problem you are trying to solve. Here are some useful guides on [loss functions](https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/) and [activation functions](https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/).
  
  Additional to the above, try to use [scaling(normalize)](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html) the inputs before feeding to the network, so that the network may generalize, as it looks like bitrate value above 300 is likely to be associated with class 1. It may improve performance.
  
  > **Comments:**
  > **username_0**: Thanks Marcus! Do you have any useful tutorial etc. advice to understand what kind of loss functions should be used with which activations or vice-versa?  
  > **username_1**: @bbasaran I updated the answer. If you are interested in a deep learning course, I would recommend Coursera's Deep Learning course.  
  
  ---
  
  