Answers.Body: <p>I had recently used a slightly unorthodox method to process such images, which involved using RNNs.</p>
  
  <p>Assume the image dimensions to be (16000, 120, 16) = (width, height, channels), as in the question.</p>
  
  <p>Apply a 2D convolution (or multiple such convolutions) of shape(1, k, c),
  such that the output of the convolutions becomes (16000, 1, c). So if you only use a single convolutional layer, <code>k=120</code>.</p>
  
  <p>Then, squeeze the extra dimension, to get the shape (16000, c). </p>
  
  <p>The problem has now been transformed back into a sequence problem! You can use RNN variants for further processing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-06T11:58:10.537Z
Answers.Text: This defeats the purpose of the transform; see updated question.
Answers.UserDisplayName: OverLordGoldDragon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-06T03:53:15.877Z
Answers.LastActivityDate: 2020-04-06T04:00:53.443Z
Answers.LastEditDate: 2020-04-06T04:00:53.443Z
Answers.UserDisplayName: Susmit Agrawal
Body: <p>After transforming timeseries into an image format, I get a width-height ratio of ~135. Typical image CNN applications involve either square or reasonably-rectangular proportions - whereas mine look nearly like lines:</p>
  
  <p><img src="https://i.sstatic.net/QHR8i.png" width="600"></p>
  
  <p>Example dimensions: <code>(16000, 120, 16) = (width, height, channels)</code>.</p>
  
  <p>Are 2D CNNs expected to work well with such aspect ratios? What hyperparameters are appropriate - namely, in Keras/TF terms, <code>strides</code>, <code>kernel_size</code> (is 'unequal' preferred, e.g. <code>strides=(16, 1)</code>)? Relevant publications would help.</p>
  
  <hr>
  
  <p><strong>Clarification</strong>: <code>width == timesteps</code>. The images are obtained via a <em>transform</em> of the timeseries, e.g. Short-time Fourier Transform. <code>channels</code> are the original channels. <code>height</code> is the result of the transform, e.g. frequency information. The task is binary classification of EEG data (w/ sigmoid output).</p>
  
  <p><a href="https://www.reddit.com/r/MachineLearning/comments/fvigee/d_can_cnns_handle_extremely_long_images/" rel="nofollow noreferrer">Relevant thread</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T11:51:53.633Z
Comments.Text: Are the spatial correlations in the vertical (`height`) dimension relevant? I.e. does the vertical distance have physical meaning?
Comments.UserDisplayName: MPA
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T11:58:59.943Z
Comments.Text: @MPA Question updated
Comments.UserDisplayName: OverLordGoldDragon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T12:07:52.987Z
Comments.Text: What is the type of the desired output? A class label (e.g. for detection), a time series (e.g. for filtering), etc.?
Comments.UserDisplayName: MPA
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T12:09:05.097Z
Comments.Text: @MPA Binary classification, updated again
Comments.UserDisplayName: OverLordGoldDragon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T14:00:14.183Z
Comments.Text: Depending on the physical nature of the input data, you could first perform 1D convolutions with dimensionality reduction along the time axis to extract the most relevant features for each frequency band, and then combine the features of each frequency band into a class prediction. Or if the data are sufficiently smooth, you could try an auto-encoder to compress the time axis into a smaller latent space representation, and perform additional analysis on the latent space.
Comments.UserDisplayName: MPA
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T14:05:51.903Z
Comments.Text: @MPA 16000 is precisely from the autoencoder approach; the full original timeseries spans 200,000+ timesteps. Though compressing it even further is unrealistic without loss of information - and the timeseries transform used is promising for discrimitative feature extraction. The linked thread has some sound suggestions I might summarize in an answer.
Comments.UserDisplayName: OverLordGoldDragon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-06T14:15:35.520Z
Comments.Text: I suppose the EEG data is quasi-periodic. What about Dynamic Mode Decomposition (DMD) and performing the classification on the dominant modes?
Comments.UserDisplayName: MPA
Tags: <convolutional-neural-networks><tensorflow><python><keras><image-processing>
Title: How to handle extremely 'long' images?
UserDisplayName: OverLordGoldDragon
ThreadText: # How to handle extremely 'long' images?
  
  (Asked by: username_0 on 2020-04-05)
  
  After transforming timeseries into an image format, I get a width-height ratio of ~135. Typical image CNN applications involve either square or reasonably-rectangular proportions - whereas mine look nearly like lines:
  
  ![](https://i.sstatic.net/QHR8i.png)
  
  Example dimensions: `(16000, 120, 16) = (width, height, channels)`.
  
  Are 2D CNNs expected to work well with such aspect ratios? What hyperparameters are appropriate - namely, in Keras/TF terms, `strides`, `kernel_size` (is 'unequal' preferred, e.g. `strides=(16, 1)`)? Relevant publications would help.
  
  ---
  
  **Clarification**: `width == timesteps`. The images are obtained via a *transform* of the timeseries, e.g. Short-time Fourier Transform. `channels` are the original channels. `height` is the result of the transform, e.g. frequency information. The task is binary classification of EEG data (w/ sigmoid output).
  
  [Relevant thread](https://www.reddit.com/r/MachineLearning/comments/fvigee/d_can_cnns_handle_extremely_long_images/)
  
  **Tags:** convolutional-neural-networks, tensorflow, python, keras, image-processing
  > **Comments:**
  > **username_1**: Are the spatial correlations in the vertical (`height`) dimension relevant? I.e. does the vertical distance have physical meaning?  
  > **username_0**: @MPA Question updated  
  > **username_1**: What is the type of the desired output? A class label (e.g. for detection), a time series (e.g. for filtering), etc.?  
  > **username_0**: @MPA Binary classification, updated again  
  > **username_1**: Depending on the physical nature of the input data, you could first perform 1D convolutions with dimensionality reduction along the time axis to extract the most relevant features for each frequency band, and then combine the features of each frequency band into a class prediction. Or if the data are sufficiently smooth, you could try an auto-encoder to compress the time axis into a smaller latent space representation, and perform additional analysis on the latent space.  
  > **username_0**: @MPA 16000 is precisely from the autoencoder approach; the full original timeseries spans 200,000+ timesteps. Though compressing it even further is unrealistic without loss of information - and the timeseries transform used is promising for discrimitative feature extraction. The linked thread has some sound suggestions I might summarize in an answer.  
  > **username_1**: I suppose the EEG data is quasi-periodic. What about Dynamic Mode Decomposition (DMD) and performing the classification on the dominant modes?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-04-06. Score: 0
  
  I had recently used a slightly unorthodox method to process such images, which involved using RNNs.
  
  Assume the image dimensions to be (16000, 120, 16) = (width, height, channels), as in the question.
  
  Apply a 2D convolution (or multiple such convolutions) of shape(1, k, c),
  such that the output of the convolutions becomes (16000, 1, c). So if you only use a single convolutional layer, `k=120`.
  
  Then, squeeze the extra dimension, to get the shape (16000, c).
  
  The problem has now been transformed back into a sequence problem! You can use RNN variants for further processing.
  
  > **Comments:**
  > **username_0**: This defeats the purpose of the transform; see updated question.  
  
  ---
  
  