Body: <blockquote>
    <p>UPDATE: The tables look messed up so i put them on pastebin for better
    visibility. <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a></p>
  </blockquote>
  
  <p>I am using Neural Network with different learning types (for example Standard Backpropagation) to classify trends in time series. As stated in several papers, data normalization is a very important factor for successful / efficient learning. I am trying to be clear and precise as possible in the description.</p>
  
  <p><strong>Problem / Learning Goal:</strong></p>
  
  <p>The network gets trained with time series and 2 indicators to predict a specific cluster.  Here is a very simple (madeup) example of raw data to understand the problem:</p>
  
  <p><strong>Example RAW Data</strong>
  Timestamp;DensityX;WaveLengthY;Temperature (K)</p>
  
  <p>1;0.1;2;200</p>
  
  <p>2;0.9;3;150</p>
  
  <p>3;-0.5;1;175</p>
  
  <p>4;0;6;154</p>
  
  <p>5;1;8;155</p>
  
  <p>6;1.3;1.5;220</p>
  
  <p>7;-0.5;3.4;250</p>
  
  <p>8;0.2;2;255</p>
  
  <p>9;0.1;1;180</p>
  
  <blockquote>
    <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
  </blockquote>
  
  <p>I use the following process to generate suitable sample data for training:</p>
  
  <p>The neural network receives n time slices with the indicators and tries to check if a future trend in the temperature occurs (for x future time slices).
  For example n = 2; x=3. </p>
  
  <p>The input and output are defined as follows:</p>
  
  <p><strong>Input vector:</strong></p>
  
  <ul>
  <li>In1 =  Density_(t-2) </li>
  <li>In2 =  Wavelength_(t-2) </li>
  <li>In3 = Density_(t-1) </li>
  <li>In4 = Wavelength_(t-1)</li>
  </ul>
  
  <p><strong>Output vector:</strong></p>
  
  <p>Output Vector is a classification encoded by Effects Encoding or Dummy Encoding (Details in “Neural Networks using C# Succinctly”)</p>
  
  <p>Calculation:</p>
  
  <ul>
  <li>Classification “Down” :  Temperature drops 3 times in a row (Encoded as 0;1)</li>
  <li>Classification: “Stable”:    Temperature does neither drop nor raises 3 times in a row (1;0)</li>
  <li>Classification: “Up”:    Temperature raises 3 times in a row. (-1;-1;)</li>
  </ul>
  
  <p>So the “processed” training sample would look like this:</p>
  
  <p><strong>Processed Data</strong></p>
  
  <p>Pattern;I1;I2;I3;I4;O1;O2;Class;Used TS</p>
  
  <p>1;0.1;2;0.9;3;0;1;Down;1 to 5</p>
  
  <p>2;0.9;3;-0.5;1;-1;-1;Up;2 to 6</p>
  
  <p>3;-0.5;1;0;6;-1;-1;Up;3 to 7</p>
  
  <p>4;0;6;1;8;1;0;Stable;4 to 8</p>
  
  <p>5;1;8;1.3;1.5;1;0;Stable;5 to 9</p>
  
  <blockquote>
    <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
  </blockquote>
  
  <p>As you can see due to the different indicators ranges I want to normalize the data.</p>
  
  <p>Basically I found the following propositions in literature and research:</p>
  
  <p><strong>Min/Max Normalization</strong></p>
  
  <p>Requires the following values to calculate 
   - dataHigh: The highest unnormalized observation.</p>
  
  <ul>
  <li><p>dataLow: The lowest unnormalized observation.</p></li>
  <li><p>normalizedHigh: The high end of the range to which the data will be normalized.</p></li>
  <li><p>normalizedLow: The low end of the range to which the data will be normalized.</p></li>
  </ul>
  
  <p><strong>Reciprocal normalization</strong></p>
  
  <p>Every value is processed to its reciprocal (x=1/x). Calculated values for density x would be:</p>
  
  <p>Timestamp;Reciprocal Density</p>
  
  <p>1;10</p>
  
  <p>2;1.111111111</p>
  
  <p>3;-2</p>
  
  <p>4;#DIV/0!</p>
  
  <p>5;1</p>
  
  <p>6;0.769230769</p>
  
  <p>7;-2</p>
  
  <p>8;5</p>
  
  <p>9;10</p>
  
  <blockquote>
    <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
  </blockquote>
  
  <p><strong>Percentage normalization</strong></p>
  
  <p>The percentual delta is calculated using the value from the previous time stamp.</p>
  
  <p>The starting point was Timestamp 1 where the Delta equals 0. 
  For each timestamp the delta percentage is calculated evaluating the previous value. So calculating the time series delta percentages would turn out to:</p>
  
  <p>Timestamp;"Delta Density X"</p>
  
  <p>1;0</p>
  
  <p>2;0.9</p>
  
  <p>3;-0.555555556</p>
  
  <p>4;0</p>
  
  <p>5;#DIV/0!</p>
  
  <p>6;1.3</p>
  
  <p>7;-0.384615385</p>
  
  <p>8;-0.4</p>
  
  <p>9;0.5</p>
  
  <blockquote>
    <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
  </blockquote>
  
  <p><strong>As you can see there are errors with handling zero values and the range is still a problem in my opinion. The Min/Max approach is generally leads to a good normalization but I think there is a problem as well, because live data may breach the max and min values of the training set.</strong></p>
  
  <p>My questions are:</p>
  
  <ul>
  <li>What are your thoughts about the general idea how I process the raw data?</li>
  <li><p>How would you normalize the given data – if at all?</p>
  
  <p>a) Does it make sense for MinMax Normalization to propose a min max value which will include live data (And throw some error in case it happens)</p>
  
  <p>b) How to handle 0 values (maybe convert it to a small positive or negative value?</p></li>
  <li><p>Are there other ideas or concepts to conduct this problem?</p></li>
  </ul>
  
  <p>I am looking forward to your input. Everything is appreciated. Thanks in advance! I also apologize for errors in the example values. Anyways, thanks for your time.</p>
  
  <p>Cheers, hob.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-03-16T18:33:32.797Z
Comments.Text: Can you please correct up your errors in the question body?
Comments.UserDisplayName: quintumnia
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-03-16T19:46:49.140Z
Comments.Text: I updated to csv-semic-sep.
Comments.UserDisplayName: hobohak
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-03-17T11:13:29.690Z
Comments.Text: Thank you for the input DuttA. It is none of the problems you have mentioned. What do you mean by increasing the precision in regards of the questions?\
Comments.UserDisplayName: hobohak
Tags: <neural-networks><training><datasets>
Title: Classification Learning - Normalization of time series and live usage
UserDisplayName: hobohak
ThreadText: # Classification Learning - Normalization of time series and live usage
  
  (Asked by: username_0 on 2018-03-16)
  
  > UPDATE: The tables look messed up so i put them on pastebin for better
  > visibility. <https://pastebin.com/gDX28uVF>
  
  I am using Neural Network with different learning types (for example Standard Backpropagation) to classify trends in time series. As stated in several papers, data normalization is a very important factor for successful / efficient learning. I am trying to be clear and precise as possible in the description.
  
  **Problem / Learning Goal:**
  
  The network gets trained with time series and 2 indicators to predict a specific cluster. Here is a very simple (madeup) example of raw data to understand the problem:
  
  **Example RAW Data**
  Timestamp;DensityX;WaveLengthY;Temperature (K)
  
  1;0.1;2;200
  
  2;0.9;3;150
  
  3;-0.5;1;175
  
  4;0;6;154
  
  5;1;8;155
  
  6;1.3;1.5;220
  
  7;-0.5;3.4;250
  
  8;0.2;2;255
  
  9;0.1;1;180
  
  > see <https://pastebin.com/gDX28uVF> for better visual
  
  I use the following process to generate suitable sample data for training:
  
  The neural network receives n time slices with the indicators and tries to check if a future trend in the temperature occurs (for x future time slices).
  For example n = 2; x=3.
  
  The input and output are defined as follows:
  
  **Input vector:**
  
  * In1 = Density\_(t-2)
  * In2 = Wavelength\_(t-2)
  * In3 = Density\_(t-1)
  * In4 = Wavelength\_(t-1)
  
  **Output vector:**
  
  Output Vector is a classification encoded by Effects Encoding or Dummy Encoding (Details in “Neural Networks using C# Succinctly”)
  
  Calculation:
  
  * Classification “Down” : Temperature drops 3 times in a row (Encoded as 0;1)
  * Classification: “Stable”: Temperature does neither drop nor raises 3 times in a row (1;0)
  * Classification: “Up”: Temperature raises 3 times in a row. (-1;-1;)
  
  So the “processed” training sample would look like this:
  
  **Processed Data**
  
  Pattern;I1;I2;I3;I4;O1;O2;Class;Used TS
  
  1;0.1;2;0.9;3;0;1;Down;1 to 5
  
  2;0.9;3;-0.5;1;-1;-1;Up;2 to 6
  
  3;-0.5;1;0;6;-1;-1;Up;3 to 7
  
  4;0;6;1;8;1;0;Stable;4 to 8
  
  5;1;8;1.3;1.5;1;0;Stable;5 to 9
  
  > see <https://pastebin.com/gDX28uVF> for better visual
  
  As you can see due to the different indicators ranges I want to normalize the data.
  
  Basically I found the following propositions in literature and research:
  
  **Min/Max Normalization**
  
  Requires the following values to calculate
  - dataHigh: The highest unnormalized observation.
  
  * dataLow: The lowest unnormalized observation.
  * normalizedHigh: The high end of the range to which the data will be normalized.
  * normalizedLow: The low end of the range to which the data will be normalized.
  
  **Reciprocal normalization**
  
  Every value is processed to its reciprocal (x=1/x). Calculated values for density x would be:
  
  Timestamp;Reciprocal Density
  
  1;10
  
  2;1.111111111
  
  3;-2
  
  4;#DIV/0!
  
  5;1
  
  6;0.769230769
  
  7;-2
  
  8;5
  
  9;10
  
  > see <https://pastebin.com/gDX28uVF> for better visual
  
  **Percentage normalization**
  
  The percentual delta is calculated using the value from the previous time stamp.
  
  The starting point was Timestamp 1 where the Delta equals 0.
  For each timestamp the delta percentage is calculated evaluating the previous value. So calculating the time series delta percentages would turn out to:
  
  Timestamp;"Delta Density X"
  
  1;0
  
  2;0.9
  
  3;-0.555555556
  
  4;0
  
  5;#DIV/0!
  
  6;1.3
  
  7;-0.384615385
  
  8;-0.4
  
  9;0.5
  
  > see <https://pastebin.com/gDX28uVF> for better visual
  
  **As you can see there are errors with handling zero values and the range is still a problem in my opinion. The Min/Max approach is generally leads to a good normalization but I think there is a problem as well, because live data may breach the max and min values of the training set.**
  
  My questions are:
  
  * What are your thoughts about the general idea how I process the raw data?
  * How would you normalize the given data – if at all?
  
    a) Does it make sense for MinMax Normalization to propose a min max value which will include live data (And throw some error in case it happens)
  
    b) How to handle 0 values (maybe convert it to a small positive or negative value?
  * Are there other ideas or concepts to conduct this problem?
  
  I am looking forward to your input. Everything is appreciated. Thanks in advance! I also apologize for errors in the example values. Anyways, thanks for your time.
  
  Cheers, hob.
  
  **Tags:** neural-networks, training, datasets
  > **Comments:**
  > **username_1**: Can you please correct up your errors in the question body?  
  > **username_0**: I updated to csv-semic-sep.  
  > **username_0**: Thank you for the input DuttA. It is none of the problems you have mentioned. What do you mean by increasing the precision in regards of the questions?\  
  
  ---
  
  ## 0 answers
  
  