Answers.Body: <p>It is a common practice to encode categorical data using embedding layers.</p>
  <p>In your case, it seems like you have a stream of discrete (categorical) input and you want to use a 1d convolution (you could alternatively use an LSTM, transformer or any other time-aware architecture)</p>
  <p>The standard way to work with this data is to transform each of the discrete variables into a learned vector, which will be learned during the training of the model (i.e. an embedding layer - you can find these layers in pytorch, tensorflow and basically any other deeplearning framework).</p>
  <p>Once the variables are transformed, you have a 2D  signal which you can use your 1D convolution on.</p>
  <p>Converting the labels to integers is not going to work well with neural networks. As you said, this kind of transformation assumes order and would make it significantly more difficult for your network to learn the right function.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-13T09:04:50.103Z
Answers.LastActivityDate: 2024-07-13T09:04:50.103Z
Answers.UserDisplayName: ereisr
Answers.Body: <p>The key lies in how the categorical data is transformed into a numeric representation before being fed into the CNN. I believe what you are asking  can easily be done through one-hot encoding.</p>
  <p>Here is how it can be done without embeddings:</p>
  <p>One-Hot Encoding: Represent each category as a one-hot vector. This approach avoids imposing any ordinal relationships between categories.</p>
  <p>For example, with categories red, green, and blue, you encode them as:</p>
  <p>red: [1, 0, 0]
  green: [0, 1, 0]
  blue: [0, 0, 1]</p>
  <p>Input Transformation: If your data is a 1D time series of categorical values, transform it into a 2D input where each category is represented as its one-hot vector. The input for the CNN would then be a 2D tensor of shape (sequence_length, num_categories).</p>
  <p>Convolution Operations: A 1D CNN kernel can then slide over this 2D input. The kernel weights interact with the one-hot vectors directly. Since each vector uniquely represents a category, the model learns patterns over these vectors just as it would for numerical time series.</p>
  <p>Note:</p>
  <ol>
  <li>The convolutional operation doesn't assume ordinality in one-hot encoding, as each category is represented independently in a distinct channel.</li>
  <li>CNNs can learn spatial relationships between categories by examining combinations of one-hot vectors in local windows.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-10T21:46:42.043Z
Answers.LastActivityDate: 2024-12-10T21:46:42.043Z
Answers.UserDisplayName: Shahidul
Answers.Body: <p>This is just my opinion, but I don't think that what you are proposing can be done on categorical data, nor do I think you can meaningfully encode categorical data so that it can be a substrate for 1D CNNs.  In a 1D CNN, a 1D kernel (a vector of numbers) would slide over your 1D dataset and carry out numerical operations on that dataset.  This cannot be done with categories (one-hot encoded or not).</p>
  <p>This article has a very good and simple explanation of how 1D CNNs work:  <a href="https://towardsdatascience.com/understanding-1d-and-3d-convolution-neural-network-keras-9d8f76e29610" rel="nofollow noreferrer">Understanding 1D and 3D Convolution Neural Network | Keras</a>.</p>
  <p>The Kaggle exercise in the <a href="https://www.kaggle.com/learn/computer-vision" rel="nofollow noreferrer">Computer Vision course</a> also provides valuable information (see Lesson 4, Exercise)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-20T20:43:55.710Z
Answers.Text: Isn't 1D convolution commonly used in NLP tasks, where inputs are for example words (so they are categorical)?
Answers.UserDisplayName: NikoNyrh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-20T21:42:21.903Z
Answers.Text: @NikoNyrh, you might be right.  This [article](https://becominghuman.ai/1-dimensional-convolution-layer-for-nlp-task-5d2e86e0229c) and this [article](https://towardsdatascience.com/nlp-with-cnns-a6aa743bdc1e) seem to thinks so.  I think this is a good question and deserves an answer from someone more knowledgeable than I am.
Answers.UserDisplayName: Snehal Patel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-21T15:41:52.763Z
Answers.LastActivityDate: 2022-10-21T15:57:05.753Z
Answers.LastEditDate: 2022-10-21T15:57:05.753Z
Answers.UserDisplayName: Snehal Patel
Body: <p>Is there a way to encode categorical nominal (no ordered) data to be used in CNN models?</p>
  <p>Let's say I need to create a 1D CNN model for categorization of time series but the values are not measurements, but categories like for example colors:</p>
  <ol>
  <li>red, red, blue, green, blue, red, green</li>
  <li>blue, red, green, green, red, red, green</li>
  <li>green, blue, green, red, red, blue, red
  and so on (can be 2D array of colors input as well)</li>
  </ol>
  <p>It seems to me it would be hard to one-hot encode it. On the other hand If I change colors into integers like: 1-red, 2-green, 3-blue, then wouldn't convolutional layers assume these are ordinal or discrete data and proceed as such?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-21T16:14:30.060Z
Comments.Text: This sounds like processing sentences where the words (categories) are in some order. It’s up for debate if a convolutional architecture would be preferred to a recurrent architecture (maybe a combination), but is absolutely can be done. [Brandon Rohrer discusses this around 23:00](https://youtube.com/watch?v=FmpDIaiMIeA), and that approach seems perfectly reasonable for your task, if you’re determined to use a convolutional architecture.
Comments.UserDisplayName: Dave
Tags: <deep-learning><convolutional-neural-networks><data-preprocessing><time-series><statistics>
Title: How to encode categorical data for a convolutional model?
UserDisplayName: GKozinski
ThreadText: # How to encode categorical data for a convolutional model?
  
  (Asked by: username_0 on 2022-10-21)
  
  Is there a way to encode categorical nominal (no ordered) data to be used in CNN models?
  
  Let's say I need to create a 1D CNN model for categorization of time series but the values are not measurements, but categories like for example colors:
  
  1. red, red, blue, green, blue, red, green
  2. blue, red, green, green, red, red, green
  3. green, blue, green, red, red, blue, red
     and so on (can be 2D array of colors input as well)
  
  It seems to me it would be hard to one-hot encode it. On the other hand If I change colors into integers like: 1-red, 2-green, 3-blue, then wouldn't convolutional layers assume these are ordinal or discrete data and proceed as such?
  
  **Tags:** deep-learning, convolutional-neural-networks, data-preprocessing, time-series, statistics
  > **Comments:**
  > **username_1**: This sounds like processing sentences where the words (categories) are in some order. It’s up for debate if a convolutional architecture would be preferred to a recurrent architecture (maybe a combination), but is absolutely can be done. [Brandon Rohrer discusses this around 23:00](https://youtube.com/watch?v=FmpDIaiMIeA), and that approach seems perfectly reasonable for your task, if you’re determined to use a convolutional architecture.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2024-07-13. Score: 0
  
  It is a common practice to encode categorical data using embedding layers.
  
  In your case, it seems like you have a stream of discrete (categorical) input and you want to use a 1d convolution (you could alternatively use an LSTM, transformer or any other time-aware architecture)
  
  The standard way to work with this data is to transform each of the discrete variables into a learned vector, which will be learned during the training of the model (i.e. an embedding layer - you can find these layers in pytorch, tensorflow and basically any other deeplearning framework).
  
  Once the variables are transformed, you have a 2D signal which you can use your 1D convolution on.
  
  Converting the labels to integers is not going to work well with neural networks. As you said, this kind of transformation assumes order and would make it significantly more difficult for your network to learn the right function.
  
  
  ---
  
  ### Answer by username_3 on 2024-12-10. Score: 0
  
  The key lies in how the categorical data is transformed into a numeric representation before being fed into the CNN. I believe what you are asking can easily be done through one-hot encoding.
  
  Here is how it can be done without embeddings:
  
  One-Hot Encoding: Represent each category as a one-hot vector. This approach avoids imposing any ordinal relationships between categories.
  
  For example, with categories red, green, and blue, you encode them as:
  
  red: [1, 0, 0]
  green: [0, 1, 0]
  blue: [0, 0, 1]
  
  Input Transformation: If your data is a 1D time series of categorical values, transform it into a 2D input where each category is represented as its one-hot vector. The input for the CNN would then be a 2D tensor of shape (sequence\_length, num\_categories).
  
  Convolution Operations: A 1D CNN kernel can then slide over this 2D input. The kernel weights interact with the one-hot vectors directly. Since each vector uniquely represents a category, the model learns patterns over these vectors just as it would for numerical time series.
  
  Note:
  
  1. The convolutional operation doesn't assume ordinality in one-hot encoding, as each category is represented independently in a distinct channel.
  2. CNNs can learn spatial relationships between categories by examining combinations of one-hot vectors in local windows.
  
  
  ---
  
  ### Answer by username_4 on 2022-10-21. Score: 0
  
  This is just my opinion, but I don't think that what you are proposing can be done on categorical data, nor do I think you can meaningfully encode categorical data so that it can be a substrate for 1D CNNs. In a 1D CNN, a 1D kernel (a vector of numbers) would slide over your 1D dataset and carry out numerical operations on that dataset. This cannot be done with categories (one-hot encoded or not).
  
  This article has a very good and simple explanation of how 1D CNNs work: [Understanding 1D and 3D Convolution Neural Network | Keras](https://towardsdatascience.com/understanding-1d-and-3d-convolution-neural-network-keras-9d8f76e29610).
  
  The Kaggle exercise in the [Computer Vision course](https://www.kaggle.com/learn/computer-vision) also provides valuable information (see Lesson 4, Exercise)
  
  > **Comments:**
  > **username_5**: Isn't 1D convolution commonly used in NLP tasks, where inputs are for example words (so they are categorical)?  
  > **username_4**: @NikoNyrh, you might be right.  This [article](https://becominghuman.ai/1-dimensional-convolution-layer-for-nlp-task-5d2e86e0229c) and this [article](https://towardsdatascience.com/nlp-with-cnns-a6aa743bdc1e) seem to thinks so.  I think this is a good question and deserves an answer from someone more knowledgeable than I am.  
  
  ---
  
  