Answers.Body: <p>Indeed crossing all 198 feature as interactions can lead to an explosion in the number of features which may not be feasible and risk overfitting. One traditional approach is feature engineering based on domain knowledge such as the distance between holds, sequences of holds, or patterns typical in climbing routes to select features.</p>
  <p>Another approach could use embedding to represent the categorical value of each hold's state, for example, you could use an embedding layer to convert the 4-dimansional categorical features of each of the 198 holds into a higher dimensional, say 8, feature space (from keras.layers import Embedding). After embedding layer, interactions between features can be captured by subsequent <em>dense</em> layers of the neural network which can learn complex, non-linear combinations of the embedded features, thus implicitly allowing the model to capture higher-order <em>interactions</em>. When successful training is done, holds that frequently co-occur in climbing routes of similar difficulty may end up with similar embeddings with the expressive power of higher dimensional continuous embedding space where subtle differences and similarities can be captured more effectively than a lower-dimensional sparse discrete space.</p>
  <p>Finally consider reshaping the embedding layer into a grid-like layer as the layout of holds has apparently a spatial aspect. Convolutional layers can then be used to extract features from this grid representation which can capture local patterns and dependencies between adjacent holds effectively. And don't forget to flatten the last convolution layer to feed into the first dense layer for the final softmax classification.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-05T01:27:39.217Z
Answers.LastActivityDate: 2024-08-05T01:27:39.217Z
Answers.UserDisplayName: cinch
Body: <p>I am working on a project to make a model using Keras to guess the difficulty of climbing routes on a standardised climbing wall (<a href="https://moonclimbing.com/moonboard" rel="nofollow noreferrer">https://moonclimbing.com/moonboard</a>). Each hold on the wall is either:</p>
  <ol start="0">
  <li>not in the route at all</li>
  <li>in the route as one of the holds used to start</li>
  <li>in the route as the final hold to finish the route</li>
  <li>in the middle of the route (between the start and end)</li>
  </ol>
  <p>The input to the model is the state of each hold on the standardised wall (whether it is in case 0, 1, 2 or 3 from above). The output will be a predicted difficulty of the climb in one of 11 categories. Since you move between holds in climbing (and the way holds are combined in a route is essential to guessing the difficulty), it seems to me that it would make a lot of sense in theory that the state of all holds on the wall should be feature-crossed together (i.e. a feature-cross with roughly 200 features). However, I haven't seen any examples which use feature-crossing of so many features, and also don't really know how this would affect the architecture of the model I should choose.
  Is it a good idea to try to cross all 198 features, or is there some different approach I should try?</p>
  
Tags: <classification><tensorflow><keras>
Title: Feature Crossing with ~200 features
UserDisplayName: seba
ThreadText: # Feature Crossing with ~200 features
  
  (Asked by: username_0 on 2024-08-04)
  
  I am working on a project to make a model using Keras to guess the difficulty of climbing routes on a standardised climbing wall (<https://moonclimbing.com/moonboard>). Each hold on the wall is either:
  
  0. not in the route at all
  1. in the route as one of the holds used to start
  2. in the route as the final hold to finish the route
  3. in the middle of the route (between the start and end)
  
  The input to the model is the state of each hold on the standardised wall (whether it is in case 0, 1, 2 or 3 from above). The output will be a predicted difficulty of the climb in one of 11 categories. Since you move between holds in climbing (and the way holds are combined in a route is essential to guessing the difficulty), it seems to me that it would make a lot of sense in theory that the state of all holds on the wall should be feature-crossed together (i.e. a feature-cross with roughly 200 features). However, I haven't seen any examples which use feature-crossing of so many features, and also don't really know how this would affect the architecture of the model I should choose.
  Is it a good idea to try to cross all 198 features, or is there some different approach I should try?
  
  **Tags:** classification, tensorflow, keras
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-08-05. Score: 0
  
  Indeed crossing all 198 feature as interactions can lead to an explosion in the number of features which may not be feasible and risk overfitting. One traditional approach is feature engineering based on domain knowledge such as the distance between holds, sequences of holds, or patterns typical in climbing routes to select features.
  
  Another approach could use embedding to represent the categorical value of each hold's state, for example, you could use an embedding layer to convert the 4-dimansional categorical features of each of the 198 holds into a higher dimensional, say 8, feature space (from keras.layers import Embedding). After embedding layer, interactions between features can be captured by subsequent *dense* layers of the neural network which can learn complex, non-linear combinations of the embedded features, thus implicitly allowing the model to capture higher-order *interactions*. When successful training is done, holds that frequently co-occur in climbing routes of similar difficulty may end up with similar embeddings with the expressive power of higher dimensional continuous embedding space where subtle differences and similarities can be captured more effectively than a lower-dimensional sparse discrete space.
  
  Finally consider reshaping the embedding layer into a grid-like layer as the layout of holds has apparently a spatial aspect. Convolutional layers can then be used to extract features from this grid representation which can capture local patterns and dependencies between adjacent holds effectively. And don't forget to flatten the last convolution layer to feed into the first dense layer for the final softmax classification.
  
  
  ---
  
  