title: MemoryError using MultinomilNB
body: <p>I got MemoryError while using <code>sklearn.naive_bayes.MultinomialNB</code> for Named Entity Recognition on a big data, where <code>train.shape = (416330, 97896)</code>.</p>
  <pre><code>data_train = pd.read_csv(path[0] + &quot;train_SENTENCED.tsv&quot;, encoding=&quot;utf-8&quot;, sep='\t', quoting=csv.QUOTE_NONE)
  data_test = pd.read_csv(path[0] + &quot;test_SENTENCED.tsv&quot;, encoding=&quot;utf-8&quot;, sep='\t', quoting=csv.QUOTE_NONE)
  
  print('TRAIN_DATA:\n', data_train.tail(5))
  
  # FIT TRANSFORM
  X_TRAIN = data_train.drop('Tag', axis=1)
  X_TEST = data_test.drop('Tag', axis=1)
  v = DictVectorizer(sparse=False)
  X_train = v.fit_transform(X_TRAIN.to_dict('records'))
  X_test = v.transform(X_TEST.to_dict('records'))
  y_train = data_train.Tag.values
  y_test = data_test.Tag.values
  classes = np.unique(y_train)
  classes = classes.tolist()
  
  nb = MultinomialNB(alpha=0.01)
  nb.partial_fit(X_train, y_train, classes)
  new_classes = classes.copy()
  new_classes.pop()
  predictions = nb.predict(X_test)
  </code></pre>
  <p>The error output is following:</p>
  <pre><code>Traceback (most recent call last):  
  File &quot;naive_bayes_classifier/main.py&quot;, line 107, in &lt;module&gt;  
        X_train = v.fit_transform(X_TRAIN.to_dict('records'))  
  File &quot;lib/python3.8/site-packages/sklearn/feature_extraction/_dict_vectorizer.py&quot;, line 313, in fit_transform  
        return self._transform(X, fitting=True)  
  File &quot;lib/python3.8/site-packages/sklearn/feature_extraction/_dict_vectorizer.py&quot;, line 282, in _transform  
        result_matrix = result_matrix.toarray()  
  File &quot;lib/python3.8/site-packages/scipy/sparse/compressed.py&quot;, line 1031, in toarray  
        out = self._process_toarray_args(order, out)  
  File &quot;lib/python3.8/site-packages/scipy/sparse/base.py&quot;, line 1202, in _process_toarray_args  
        return np.zeros(self.shape, dtype=self.dtype, order=order) 
  numpy.core._exceptions.MemoryError: Unable to allocate 304. GiB for an array with shape (416330, 97896) and data type float64
  </code></pre>
  <p>Although, I put such parameter for <code>DictVectorizer(sparse=False, dtype=np.short)</code>, the code then returns error for <code>nb.partial(X_train, y_train, classes)</code> line.</p>
  <p>How can I prevent this Memory Error? Is there a proper way to solve it? I thought about splitting the train set, but since the vectors fit the corresponding dataset, will it be correct solution?</p>