<blockquote>
  <p>UPDATE: The tables look messed up so i put them on pastebin for better
  visibility. <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a></p>
</blockquote>

<p>I am using Neural Network with different learning types (for example Standard Backpropagation) to classify trends in time series. As stated in several papers, data normalization is a very important factor for successful / efficient learning. I am trying to be clear and precise as possible in the description.</p>

<p><strong>Problem / Learning Goal:</strong></p>

<p>The network gets trained with time series and 2 indicators to predict a specific cluster.  Here is a very simple (madeup) example of raw data to understand the problem:</p>

<p><strong>Example RAW Data</strong>
Timestamp;DensityX;WaveLengthY;Temperature (K)</p>

<p>1;0.1;2;200</p>

<p>2;0.9;3;150</p>

<p>3;-0.5;1;175</p>

<p>4;0;6;154</p>

<p>5;1;8;155</p>

<p>6;1.3;1.5;220</p>

<p>7;-0.5;3.4;250</p>

<p>8;0.2;2;255</p>

<p>9;0.1;1;180</p>

<blockquote>
  <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
</blockquote>

<p>I use the following process to generate suitable sample data for training:</p>

<p>The neural network receives n time slices with the indicators and tries to check if a future trend in the temperature occurs (for x future time slices).
For example n = 2; x=3. </p>

<p>The input and output are defined as follows:</p>

<p><strong>Input vector:</strong></p>

<ul>
<li>In1 =  Density_(t-2) </li>
<li>In2 =  Wavelength_(t-2) </li>
<li>In3 = Density_(t-1) </li>
<li>In4 = Wavelength_(t-1)</li>
</ul>

<p><strong>Output vector:</strong></p>

<p>Output Vector is a classification encoded by Effects Encoding or Dummy Encoding (Details in “Neural Networks using C# Succinctly”)</p>

<p>Calculation:</p>

<ul>
<li>Classification “Down” :  Temperature drops 3 times in a row (Encoded as 0;1)</li>
<li>Classification: “Stable”:    Temperature does neither drop nor raises 3 times in a row (1;0)</li>
<li>Classification: “Up”:    Temperature raises 3 times in a row. (-1;-1;)</li>
</ul>

<p>So the “processed” training sample would look like this:</p>

<p><strong>Processed Data</strong></p>

<p>Pattern;I1;I2;I3;I4;O1;O2;Class;Used TS</p>

<p>1;0.1;2;0.9;3;0;1;Down;1 to 5</p>

<p>2;0.9;3;-0.5;1;-1;-1;Up;2 to 6</p>

<p>3;-0.5;1;0;6;-1;-1;Up;3 to 7</p>

<p>4;0;6;1;8;1;0;Stable;4 to 8</p>

<p>5;1;8;1.3;1.5;1;0;Stable;5 to 9</p>

<blockquote>
  <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
</blockquote>

<p>As you can see due to the different indicators ranges I want to normalize the data.</p>

<p>Basically I found the following propositions in literature and research:</p>

<p><strong>Min/Max Normalization</strong></p>

<p>Requires the following values to calculate 
 - dataHigh: The highest unnormalized observation.</p>

<ul>
<li><p>dataLow: The lowest unnormalized observation.</p></li>
<li><p>normalizedHigh: The high end of the range to which the data will be normalized.</p></li>
<li><p>normalizedLow: The low end of the range to which the data will be normalized.</p></li>
</ul>

<p><strong>Reciprocal normalization</strong></p>

<p>Every value is processed to its reciprocal (x=1/x). Calculated values for density x would be:</p>

<p>Timestamp;Reciprocal Density</p>

<p>1;10</p>

<p>2;1.111111111</p>

<p>3;-2</p>

<p>4;#DIV/0!</p>

<p>5;1</p>

<p>6;0.769230769</p>

<p>7;-2</p>

<p>8;5</p>

<p>9;10</p>

<blockquote>
  <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
</blockquote>

<p><strong>Percentage normalization</strong></p>

<p>The percentual delta is calculated using the value from the previous time stamp.</p>

<p>The starting point was Timestamp 1 where the Delta equals 0. 
For each timestamp the delta percentage is calculated evaluating the previous value. So calculating the time series delta percentages would turn out to:</p>

<p>Timestamp;"Delta Density X"</p>

<p>1;0</p>

<p>2;0.9</p>

<p>3;-0.555555556</p>

<p>4;0</p>

<p>5;#DIV/0!</p>

<p>6;1.3</p>

<p>7;-0.384615385</p>

<p>8;-0.4</p>

<p>9;0.5</p>

<blockquote>
  <p>see <a href="https://pastebin.com/gDX28uVF" rel="nofollow noreferrer">https://pastebin.com/gDX28uVF</a> for better visual</p>
</blockquote>

<p><strong>As you can see there are errors with handling zero values and the range is still a problem in my opinion. The Min/Max approach is generally leads to a good normalization but I think there is a problem as well, because live data may breach the max and min values of the training set.</strong></p>

<p>My questions are:</p>

<ul>
<li>What are your thoughts about the general idea how I process the raw data?</li>
<li><p>How would you normalize the given data – if at all?</p>

<p>a) Does it make sense for MinMax Normalization to propose a min max value which will include live data (And throw some error in case it happens)</p>

<p>b) How to handle 0 values (maybe convert it to a small positive or negative value?</p></li>
<li><p>Are there other ideas or concepts to conduct this problem?</p></li>
</ul>

<p>I am looking forward to your input. Everything is appreciated. Thanks in advance! I also apologize for errors in the example values. Anyways, thanks for your time.</p>

<p>Cheers, hob.</p>

CC BY-SA 3.0
2018-03-16T18:33:32.797Z
Can you please correct up your errors in the question body?
quintumnia
CC BY-SA 3.0
2018-03-16T19:46:49.140Z
I updated to csv-semic-sep.
hobohak
CC BY-SA 3.0
2018-03-17T11:13:29.690Z
Thank you for the input DuttA. It is none of the problems you have mentioned. What do you mean by increasing the precision in regards of the questions?\
hobohak
<neural-networks><training><datasets>
Classification Learning - Normalization of time series and live usage
hobohak
# Classification Learning - Normalization of time series and live usage

(Asked by: username_0 on 2018-03-16)

> UPDATE: The tables look messed up so i put them on pastebin for better
> visibility. <https://pastebin.com/gDX28uVF>

I am using Neural Network with different learning types (for example Standard Backpropagation) to classify trends in time series. As stated in several papers, data normalization is a very important factor for successful / efficient learning. I am trying to be clear and precise as possible in the description.

**Problem / Learning Goal:**

The network gets trained with time series and 2 indicators to predict a specific cluster. Here is a very simple (madeup) example of raw data to understand the problem:

**Example RAW Data**
Timestamp;DensityX;WaveLengthY;Temperature (K)

1;0.1;2;200

2;0.9;3;150

3;-0.5;1;175

4;0;6;154

5;1;8;155

6;1.3;1.5;220

7;-0.5;3.4;250

8;0.2;2;255

9;0.1;1;180

> see <https://pastebin.com/gDX28uVF> for better visual

I use the following process to generate suitable sample data for training:

The neural network receives n time slices with the indicators and tries to check if a future trend in the temperature occurs (for x future time slices).
For example n = 2; x=3.

The input and output are defined as follows:

**Input vector:**

* In1 = Density\_(t-2)
* In2 = Wavelength\_(t-2)
* In3 = Density\_(t-1)
* In4 = Wavelength\_(t-1)

**Output vector:**

Output Vector is a classification encoded by Effects Encoding or Dummy Encoding (Details in “Neural Networks using C# Succinctly”)

Calculation:

* Classification “Down” : Temperature drops 3 times in a row (Encoded as 0;1)
* Classification: “Stable”: Temperature does neither drop nor raises 3 times in a row (1;0)
* Classification: “Up”: Temperature raises 3 times in a row. (-1;-1;)

So the “processed” training sample would look like this:

**Processed Data**

Pattern;I1;I2;I3;I4;O1;O2;Class;Used TS

1;0.1;2;0.9;3;0;1;Down;1 to 5

2;0.9;3;-0.5;1;-1;-1;Up;2 to 6

3;-0.5;1;0;6;-1;-1;Up;3 to 7

4;0;6;1;8;1;0;Stable;4 to 8

5;1;8;1.3;1.5;1;0;Stable;5 to 9

> see <https://pastebin.com/gDX28uVF> for better visual

As you can see due to the different indicators ranges I want to normalize the data.

Basically I found the following propositions in literature and research:

**Min/Max Normalization**

Requires the following values to calculate
- dataHigh: The highest unnormalized observation.

* dataLow: The lowest unnormalized observation.
* normalizedHigh: The high end of the range to which the data will be normalized.
* normalizedLow: The low end of the range to which the data will be normalized.

**Reciprocal normalization**

Every value is processed to its reciprocal (x=1/x). Calculated values for density x would be:

Timestamp;Reciprocal Density

1;10

2;1.111111111

3;-2

4;#DIV/0!

5;1

6;0.769230769

7;-2

8;5

9;10

> see <https://pastebin.com/gDX28uVF> for better visual

**Percentage normalization**

The percentual delta is calculated using the value from the previous time stamp.

The starting point was Timestamp 1 where the Delta equals 0.
For each timestamp the delta percentage is calculated evaluating the previous value. So calculating the time series delta percentages would turn out to:

Timestamp;"Delta Density X"

1;0

2;0.9

3;-0.555555556

4;0

5;#DIV/0!

6;1.3

7;-0.384615385

8;-0.4

9;0.5

> see <https://pastebin.com/gDX28uVF> for better visual

**As you can see there are errors with handling zero values and the range is still a problem in my opinion. The Min/Max approach is generally leads to a good normalization but I think there is a problem as well, because live data may breach the max and min values of the training set.**

My questions are:

* What are your thoughts about the general idea how I process the raw data?
* How would you normalize the given data – if at all?

  a) Does it make sense for MinMax Normalization to propose a min max value which will include live data (And throw some error in case it happens)

  b) How to handle 0 values (maybe convert it to a small positive or negative value?
* Are there other ideas or concepts to conduct this problem?

I am looking forward to your input. Everything is appreciated. Thanks in advance! I also apologize for errors in the example values. Anyways, thanks for your time.

Cheers, hob.

**Tags:** neural-networks, training, datasets

> **Comments:**
> **username_1**: Can you please correct up your errors in the question body?  
> **username_0**: I updated to csv-semic-sep.  
> **username_0**: Thank you for the input DuttA. It is none of the problems you have mentioned. What do you mean by increasing the precision in regards of the questions?\  

---

## 0 answers

