title: How do I store a fitted PCA so that I may transpose unseen testing dataset? I do not wish to keep the large training dataset on my CPU
body: <p>I have a very large training dataset. My training dataset contains 1050 gestures, with each gesture containing 12,000 data points. Feeding our machine learning models with this many data points will result to a very slow performance and poor accuracy. As a result, I used PCA to remove irrelevant characteristics from a high-dimensional space and projected the most important features into a lower-dimensional subspace, improving classification accuracy and reducing computational time. Using PCA we have reduced 12,000 data points for each gesture to 15 PCs without compromising the information extracted from the data.</p>
  <p>In the future, I would like to store my machine learning model onto an Arduino. An Arduino is a small chip that roughly has 256KB storage. My training dataset that I use to fit the PCA to is 225MB in storage, therefore not possible.</p>
  <p>Is there a way to perform and fit PCA to my training dataset so that I can transpose my unseen testing dataset in the future on the Arduino without having to store the training dataset to my Arduino for fitting?</p>
  <p>Here is my code to fit my training dataset</p>
  <pre><code>from sklearn.preprocessing import StandardScaler
  from sklearn.decomposition import PCA
  
  transposed_normDF.columns = transposed_normDF.columns.map(str)
  features = [str(i) for i in range(0,11999)]
  x = transposed_normDF.loc[:, features].values
  y = df.loc[:,['label']].values
  
  pca = PCA(n_components=0.99)
  principalComponents = pca.fit_transform(x)
  
  pc = pca.explained_variance_ratio_.cumsum()
  x1 = StandardScaler().fit_transform(principalComponents)
  full_newdf = pd.DataFrame(data = x1
               , columns = [f'pc_stdscaled_{i}' for i in range(len(pc))])
  full_finalDf = pd.concat([full_newdf, df[['label']]], axis = 1)
  print(full_finalDf)
  print(full_newdf.shape)
  </code></pre>
  <p>Here is my code to transpose unseen data</p>
  <pre><code>pca = PCA(n_components=0.99)
  
  newdata_transformed = pca.transform(in_data)
  pc = pca.explained_variance_ratio_.cumsum()
  x1 = StandardScaler().fit(principalComponents)
  X1 = x1.transform(newdata_transformed)
  newdf = pd.DataFrame(data = X1
               , columns = [f'pc_stdscaled_{i}' for i in range(len(pc))])
  newdf.head()
  </code></pre>