Answers.Body: <p>I found this solution and will be happy about any improvements or suggestions:</p>
  
  <p>First, I create a random dataset of images, which are 28x28 pixels, and corresponding random labels (just for sake of clarification, I have another image dataset, this is just for explaining). Then I use a sklearn module for splitting the data:</p>
  
  <pre><code>import numpy as np
  from sklearn.model_selection import train_test_split
  
  
  # create the data and labels
  def pixel_dataset(n_data=10, dpi = 28):
  
      ary = np.zeros((n_data, dpi, dpi))
      label = [] 
  
      for i in np.arange(n_data):
          labels =['a', 'b', 'c']
          ary[i,::] = np.reshape(np.array(np.random.randint(0, 255, 28*28)),(28,28))
          label.append(np.random.choice(labels, 1)[0])
  
      return ary, label 
  
  
  # create the test train split    
  def mydata(test_size=0.3):
  
      X, y = pixel_dataset()       
      X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = test_size) 
      return (X_train, y_train), (X_test, y_test)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-28T22:58:38.247Z
Answers.LastActivityDate: 2018-10-28T22:58:38.247Z
Answers.UserDisplayName: giotto
Body: <p>I am creating a dataset made of many images which are created by preprocessing a long time series. Each image is an array of (128,128) and the there are four classes. I would like to build a dataset similar to the MNIST in scikit-learn.database  but I have no idea how to do it. </p>
  
  <p>My aim is to have something that I can call like this:</p>
  
  <pre><code>(x_train, y_train), (x_test, y_test) = my_data()
  </code></pre>
  
  <p>Should I save them as figures? or as csv? 
  Which is the best way to implement this? </p>
  
Tags: <convolutional-neural-networks><datasets>
Title: Best way to create an image dataset for CNN
UserDisplayName: giotto
ThreadText: # Best way to create an image dataset for CNN
  
  (Asked by: username_0 on 2018-10-23)
  
  I am creating a dataset made of many images which are created by preprocessing a long time series. Each image is an array of (128,128) and the there are four classes. I would like to build a dataset similar to the MNIST in scikit-learn.database  but I have no idea how to do it. 
  My aim is to have something that I can call like this:
  (x_train, y_train), (x_test, y_test) = my_data()
  
  Should I save them as figures? or as csv? 
  Which is the best way to implement this?
  
  **Tags:** convolutional-neural-networks, datasets
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2018-10-28. Score: 0 | Accepted answer
  
  I found this solution and will be happy about any improvements or suggestions:
  First, I create a random dataset of images, which are 28x28 pixels, and corresponding random labels (just for sake of clarification, I have another image dataset, this is just for explaining). Then I use a sklearn module for splitting the data:
  import numpy as np
  from sklearn.model_selection import train_test_split
  
  
  # create the data and labels
  def pixel_dataset(n_data=10, dpi = 28):
  
      ary = np.zeros((n_data, dpi, dpi))
      label = [] 
  
      for i in np.arange(n_data):
          labels =['a', 'b', 'c']
          ary[i,::] = np.reshape(np.array(np.random.randint(0, 255, 28*28)),(28,28))
          label.append(np.random.choice(labels, 1)[0])
  
      return ary, label 
  
  
  # create the test train split    
  def mydata(test_size=0.3):
  
      X, y = pixel_dataset()       
      X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = test_size) 
      return (X_train, y_train), (X_test, y_test)
  
  
  ---
  
  