Answers.Body: <blockquote>
  <p>Or is there no clear answer and would this be something I'd only be able to figure out by testing against data?</p>
  </blockquote>
  <p>That is the general rule you should always consider when looking at feature engineering (which is what you are proposing), as well as for many architecture choices.</p>
  <p>It is very hard to tell in advance what a change to a machine learning system will do for whichever metrics you are interested in. You may have some experience that applies, or find similar experiments online that you can take inspiration from. But you will want to test everything, and should take care to use good practice when evaluating different options - e.g. a cross validation dataset (sometimes called a development dataset).</p>
  <blockquote>
  <p>Would it be redundant/not ideal if I had another feature that was the average of the past 40 opens for example? What about the max/min of the past opens?</p>
  </blockquote>
  <p>One aspect of multi-layer neural networks, is that they can <em>in theory</em> learn useful internal features in the hidden layers from raw data. These internal features are unlikely to be exact copies of mean values or min/max values, or anything else you would construct manually. However, they can be similar enough in end result that manually derived features that you think of will not make much difference.</p>
  <p>So you would think that derived features would not be useful in nerual networks. In practice though they can be, because the convergence process to find the best internal features is not perfect. Smart feature engineering can improve the performance of a neural network classification or regression supervised learning. Sometimes you can find &quot;golden&quot; engineered features that relate really well to your target variable, and that boost results significantly.</p>
  <p>A couple of things to bear in mind:</p>
  <ul>
  <li><p>A &quot;scattergun&quot; approach of trying a large number of derived features might seem attractive, but there is a risk of overfitting the training data. If you try enough times you may find something that works purely by chance but only for the training data set.</p>
  </li>
  <li><p>Nonlinear combinations that make conceptual sense given the problem domain can be worth looking at. For instance if you want to predict house prices, and your raw data was house <code>width</code> and <code>depth</code>, then floor area <code>width * depth</code> might be a useful feature.</p>
  </li>
  </ul>
  <p>Feature engineering is still something of an artform. Automated systems using the scattergun approach with filtering are competitive, but domain insight can still win.</p>
  <p>If you have vast amounts of data and the CPU time cost is not an issue, you may want to forgo feature engineering due to the theoretical redundancy. It seems possible to make a giant neural network using latest features such as skip connections and batch normalisation, feed it raw (but normalised) data, and press &quot;go&quot; to get a state-of-the-art result. From that perspective, feature engineering is for when you don't have &quot;big data&quot; or deep pockets for heavy processing - for many of us that still means feature engineering is a standard approach on every project.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-29T03:11:07.300Z
Answers.Text: +1. my empirical study even showed that derived features can replace raw features (empirical results are not affected, and even improving, when raw features are omitted). I use derived features that are intuitively useful/impactful in my domain, e.g., instead of raw coordinates of points, I use distance between points. instead of only raw price, maybe the derived feature can also be a feature of how close the last few stock prices to a shape. that's a thing right?
Answers.UserDisplayName: Sanyou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-29T04:48:19.807Z
Answers.Text: @Sanyou Yes, if you have good insight into your problem, making derived features using your knowledge can work very well, especially if you have smaller datasets. With stock trading, some of the "shape" rules may not be as useful as people hope though - there's a lot of pseudoscience and wishful thinking that you can have a "system" that will gain money. I don't trade stocks though, so I cannot really say which ideas are worth making derived features for in practice.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-08T20:27:07.247Z
Answers.LastActivityDate: 2021-06-08T21:30:25.400Z
Answers.LastEditDate: 2021-06-08T21:30:25.400Z
Answers.UserDisplayName: Neil Slater
Body: <p>I'm trying to create a neural network that would able to look at the current price of a crypto asset and classify between a &quot;BUY&quot;, &quot;SELL&quot; or &quot;HOLD&quot;. So far for my input features, I've decided to go with the past 40 opens, closes, highs, lows, turnover, and volumes (240 features + the current price so 241 total features).</p>
  <p>Would it be redundant/not ideal if I had another feature that was the average of the past 40 opens for example? What about the max/min of the past opens?</p>
  <p>My thinking was that with only the raw prices data of the past 40 days, the neural network would be able to &quot;detect&quot; and create the most optimum features like the average or max in the hidden layers. And therefore, having the avg. or the max/min of some existing features would be unnecessary or perhaps worsen the performance of the model.</p>
  <p>Or is there no clear answer and would this be something I'd only be able to figure out by testing against data?</p>
  <p>Thanks for your help!!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-08T16:00:18.477Z
Comments.Text: Hi Neil. Ah okay, sorry about that. I edited the question down to exclude the last bit.
Comments.UserDisplayName: Arthur Song
Tags: <feature-selection><algorithmic-trading>
Title: Selecting features for a neural network: is it redundant to have a feature that is an average (or max, or min) of some other features
UserDisplayName: Arthur Song
ThreadText: # Selecting features for a neural network: is it redundant to have a feature that is an average (or max, or min) of some other features
  
  (Asked by: username_0 on 2021-06-07)
  
  I'm trying to create a neural network that would able to look at the current price of a crypto asset and classify between a "BUY", "SELL" or "HOLD". So far for my input features, I've decided to go with the past 40 opens, closes, highs, lows, turnover, and volumes (240 features + the current price so 241 total features).
  
  Would it be redundant/not ideal if I had another feature that was the average of the past 40 opens for example? What about the max/min of the past opens?
  
  My thinking was that with only the raw prices data of the past 40 days, the neural network would be able to "detect" and create the most optimum features like the average or max in the hidden layers. And therefore, having the avg. or the max/min of some existing features would be unnecessary or perhaps worsen the performance of the model.
  
  Or is there no clear answer and would this be something I'd only be able to figure out by testing against data?
  
  Thanks for your help!!
  
  **Tags:** feature-selection, algorithmic-trading
  > **Comments:**
  > **username_0**: Hi Neil. Ah okay, sorry about that. I edited the question down to exclude the last bit.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-06-08. Score: 1
  
  > Or is there no clear answer and would this be something I'd only be able to figure out by testing against data?
  
  That is the general rule you should always consider when looking at feature engineering (which is what you are proposing), as well as for many architecture choices.
  
  It is very hard to tell in advance what a change to a machine learning system will do for whichever metrics you are interested in. You may have some experience that applies, or find similar experiments online that you can take inspiration from. But you will want to test everything, and should take care to use good practice when evaluating different options - e.g. a cross validation dataset (sometimes called a development dataset).
  
  > Would it be redundant/not ideal if I had another feature that was the average of the past 40 opens for example? What about the max/min of the past opens?
  
  One aspect of multi-layer neural networks, is that they can *in theory* learn useful internal features in the hidden layers from raw data. These internal features are unlikely to be exact copies of mean values or min/max values, or anything else you would construct manually. However, they can be similar enough in end result that manually derived features that you think of will not make much difference.
  
  So you would think that derived features would not be useful in nerual networks. In practice though they can be, because the convergence process to find the best internal features is not perfect. Smart feature engineering can improve the performance of a neural network classification or regression supervised learning. Sometimes you can find "golden" engineered features that relate really well to your target variable, and that boost results significantly.
  
  A couple of things to bear in mind:
  
  * A "scattergun" approach of trying a large number of derived features might seem attractive, but there is a risk of overfitting the training data. If you try enough times you may find something that works purely by chance but only for the training data set.
  * Nonlinear combinations that make conceptual sense given the problem domain can be worth looking at. For instance if you want to predict house prices, and your raw data was house `width` and `depth`, then floor area `width * depth` might be a useful feature.
  
  Feature engineering is still something of an artform. Automated systems using the scattergun approach with filtering are competitive, but domain insight can still win.
  
  If you have vast amounts of data and the CPU time cost is not an issue, you may want to forgo feature engineering due to the theoretical redundancy. It seems possible to make a giant neural network using latest features such as skip connections and batch normalisation, feed it raw (but normalised) data, and press "go" to get a state-of-the-art result. From that perspective, feature engineering is for when you don't have "big data" or deep pockets for heavy processing - for many of us that still means feature engineering is a standard approach on every project.
  
  > **Comments:**
  > **username_2**: +1. my empirical study even showed that derived features can replace raw features (empirical results are not affected, and even improving, when raw features are omitted). I use derived features that are intuitively useful/impactful in my domain, e.g., instead of raw coordinates of points, I use distance between points. instead of only raw price, maybe the derived feature can also be a feature of how close the last few stock prices to a shape. that's a thing right?  
  > **username_1**: @Sanyou Yes, if you have good insight into your problem, making derived features using your knowledge can work very well, especially if you have smaller datasets. With stock trading, some of the "shape" rules may not be as useful as people hope though - there's a lot of pseudoscience and wishful thinking that you can have a "system" that will gain money. I don't trade stocks though, so I cannot really say which ideas are worth making derived features for in practice.  
  
  ---
  
  