<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="nosetests sklearn --exe
.............................................................../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/manifold/spectral_embedding.py:225: UserWarning: Graph is not fully connected, spectral embedding may not works as expected.
  warnings.warn(&quot;Graph is not fully connected, spectral embedding&quot;
...........SS....../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/datasets/tests/test_base.py:124: UserWarning: Could not load sample images, PIL is not available.
  warnings.warn(&quot;Could not load sample images, PIL is not available.&quot;)
.../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/datasets/tests/test_base.py:145: UserWarning: Could not load sample images, PIL is not available.
  warnings.warn(&quot;Could not load sample images, PIL is not available.&quot;)
./usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/datasets/tests/test_base.py:161: UserWarning: Could not load sample images, PIL is not available.
  warnings.warn(&quot;Could not load sample images, PIL is not available.&quot;)
.....SS................................................S.........................................................S.........................................SSS....................../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/externals/joblib/test/test_func_inspect.py:122: UserWarning: Cannot inspect object &lt;functools.partial object at 0x6e36f18&gt;, ignore list will not work.
  nose.tools.assert_equal(filter_args(ff, ['y'], (1, )),
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F............................................................................SSS....S....S...................................................................................................................................
======================================================================
FAIL: sklearn.tests.test_common.test_regressors_train
----------------------------------------------------------------------
Traceback (most recent call last):
  File &quot;/usr/local/lib/python2.7/site-packages/nose-1.2.1-py2.7.egg/nose/case.py&quot;, line 197, in runTest
    self.test(*self.arg)
  File &quot;/usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/tests/test_common.py&quot;, line 655, in test_regressors_train
    assert_true(succeeded)
AssertionError: False is not true
-------------------- &gt;&gt; begin captured stdout &lt;&lt; ---------------------
CCA(copy=True, max_iter=500, n_components=2, scale=True, tol=1e-06)
singular matrix


--------------------- &gt;&gt; end captured stdout &lt;&lt; ----------------------

----------------------------------------------------------------------
Ran 1598 tests in 81.829s

FAILED (SKIP=15, failures=1)"><pre class="notranslate"><code class="notranslate">nosetests sklearn --exe
.............................................................../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/manifold/spectral_embedding.py:225: UserWarning: Graph is not fully connected, spectral embedding may not works as expected.
  warnings.warn("Graph is not fully connected, spectral embedding"
...........SS....../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/datasets/tests/test_base.py:124: UserWarning: Could not load sample images, PIL is not available.
  warnings.warn("Could not load sample images, PIL is not available.")
.../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/datasets/tests/test_base.py:145: UserWarning: Could not load sample images, PIL is not available.
  warnings.warn("Could not load sample images, PIL is not available.")
./usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/datasets/tests/test_base.py:161: UserWarning: Could not load sample images, PIL is not available.
  warnings.warn("Could not load sample images, PIL is not available.")
.....SS................................................S.........................................................S.........................................SSS....................../usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/externals/joblib/test/test_func_inspect.py:122: UserWarning: Cannot inspect object &lt;functools.partial object at 0x6e36f18&gt;, ignore list will not work.
  nose.tools.assert_equal(filter_args(ff, ['y'], (1, )),
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F............................................................................SSS....S....S...................................................................................................................................
======================================================================
FAIL: sklearn.tests.test_common.test_regressors_train
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose-1.2.1-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/local/lib/python2.7/site-packages/scikit_learn-0.13.1-py2.7-linux-x86_64.egg/sklearn/tests/test_common.py", line 655, in test_regressors_train
    assert_true(succeeded)
AssertionError: False is not true
-------------------- &gt;&gt; begin captured stdout &lt;&lt; ---------------------
CCA(copy=True, max_iter=500, n_components=2, scale=True, tol=1e-06)
singular matrix


--------------------- &gt;&gt; end captured stdout &lt;&lt; ----------------------

----------------------------------------------------------------------
Ran 1598 tests in 81.829s

FAILED (SKIP=15, failures=1)
</code></pre></div>
<p dir="auto">Is the failures a big deal ? Can I use it anyway ? I want to use sklearn to do logistic regression . Thanks.</p>
<p dir="auto">Currently, the <code class="notranslate">Imputer</code> works on columns. Let's assume I have a dataset with mixed categorical and numerical data points and missing values and I want to use the pipeline object:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import numpy as np
import scipy.stats
import sklearn.pipeline
import sklearn.preprocessing
import sklearn.ensemble

categorical_data = np.random.randint(0, 5, size=(10, 4))
numerical_data = np.random.randn(10, 4)
data = np.hstack((categorical_data, numerical_data))

# Add missing values
data[0, 1] = np.NaN
data[1, 5] = np.NaN

X = data
y = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1])"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-k">import</span> <span class="pl-s1">scipy</span>.<span class="pl-s1">stats</span>
<span class="pl-k">import</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">pipeline</span>
<span class="pl-k">import</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">preprocessing</span>
<span class="pl-k">import</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">ensemble</span>

<span class="pl-s1">categorical_data</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">randint</span>(<span class="pl-c1">0</span>, <span class="pl-c1">5</span>, <span class="pl-s1">size</span><span class="pl-c1">=</span>(<span class="pl-c1">10</span>, <span class="pl-c1">4</span>))
<span class="pl-s1">numerical_data</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">randn</span>(<span class="pl-c1">10</span>, <span class="pl-c1">4</span>)
<span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">hstack</span>((<span class="pl-s1">categorical_data</span>, <span class="pl-s1">numerical_data</span>))

<span class="pl-c"># Add missing values</span>
<span class="pl-s1">data</span>[<span class="pl-c1">0</span>, <span class="pl-c1">1</span>] <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-v">NaN</span>
<span class="pl-s1">data</span>[<span class="pl-c1">1</span>, <span class="pl-c1">5</span>] <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-v">NaN</span>

<span class="pl-v">X</span> <span class="pl-c1">=</span> <span class="pl-s1">data</span>
<span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">array</span>([<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>])</pre></div>
<p dir="auto">If I now try to use imputation, I can only choose a single strategy for both data types:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pipeline = sklearn.pipeline.Pipeline((('imp',
                                       sklearn.preprocessing.Imputer(
                                           strategy='most_frequent')),
                                      ('ohe',
                                       sklearn.preprocessing.OneHotEncoder(
                                           categorical_features=[0, 1, 2, 3])),
                                      ('rf',
                                       sklearn.ensemble.RandomForestClassifier())))"><pre class="notranslate"><span class="pl-s1">pipeline</span> <span class="pl-c1">=</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">pipeline</span>.<span class="pl-v">Pipeline</span>(((<span class="pl-s">'imp'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">preprocessing</span>.<span class="pl-v">Imputer</span>(
                                           <span class="pl-s1">strategy</span><span class="pl-c1">=</span><span class="pl-s">'most_frequent'</span>)),
                                      (<span class="pl-s">'ohe'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">preprocessing</span>.<span class="pl-v">OneHotEncoder</span>(
                                           <span class="pl-s1">categorical_features</span><span class="pl-c1">=</span>[<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>])),
                                      (<span class="pl-s">'rf'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">ensemble</span>.<span class="pl-v">RandomForestClassifier</span>())))</pre></div>
<p dir="auto">which would result in a more or less random value being picked for the missing value of the continuous feature:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="Xt, _ = pipeline._pre_transform(X, y)
# Now uses the return value of scipy.stats.mode which makes only very little
# sense for a continuous value
print(scipy.stats.mode(data)[5])
print(Xt.toarray()[1])"><pre class="notranslate"><span class="pl-v">Xt</span>, <span class="pl-s1">_</span> <span class="pl-c1">=</span> <span class="pl-s1">pipeline</span>.<span class="pl-en">_pre_transform</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>)
<span class="pl-c"># Now uses the return value of scipy.stats.mode which makes only very little</span>
<span class="pl-c"># sense for a continuous value</span>
<span class="pl-en">print</span>(<span class="pl-s1">scipy</span>.<span class="pl-s1">stats</span>.<span class="pl-en">mode</span>(<span class="pl-s1">data</span>)[<span class="pl-c1">5</span>])
<span class="pl-en">print</span>(<span class="pl-v">Xt</span>.<span class="pl-en">toarray</span>()[<span class="pl-c1">1</span>])</pre></div>
<p dir="auto">To overcome this, I propose to add a new attribute to the Imputer which allows to specify the columns to be imputed in order to allow something like this:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pipeline = sklearn.pipeline.Pipeline((('imp_cat',
                                       sklearn.preprocessing.Imputer(
                                           strategy='most_frequent', 
                                           columns=[0, 1, 2, 3])),
                                      ('imp_num',
                                       sklearn.preprocessing.Imputer(
                                           strategy='median',
                                           columns=[4, 5, 6, 7])),
                                      ('ohe',
                                       sklearn.preprocessing.OneHotEncoder(
                                           categorical_features=[0, 1, 2, 3])),
                                      ('rf',
                                       sklearn.ensemble.RandomForestClassifier())))"><pre class="notranslate"><span class="pl-s1">pipeline</span> <span class="pl-c1">=</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">pipeline</span>.<span class="pl-v">Pipeline</span>(((<span class="pl-s">'imp_cat'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">preprocessing</span>.<span class="pl-v">Imputer</span>(
                                           <span class="pl-s1">strategy</span><span class="pl-c1">=</span><span class="pl-s">'most_frequent'</span>, 
                                           <span class="pl-s1">columns</span><span class="pl-c1">=</span>[<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>])),
                                      (<span class="pl-s">'imp_num'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">preprocessing</span>.<span class="pl-v">Imputer</span>(
                                           <span class="pl-s1">strategy</span><span class="pl-c1">=</span><span class="pl-s">'median'</span>,
                                           <span class="pl-s1">columns</span><span class="pl-c1">=</span>[<span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>, <span class="pl-c1">7</span>])),
                                      (<span class="pl-s">'ohe'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">preprocessing</span>.<span class="pl-v">OneHotEncoder</span>(
                                           <span class="pl-s1">categorical_features</span><span class="pl-c1">=</span>[<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>])),
                                      (<span class="pl-s">'rf'</span>,
                                       <span class="pl-s1">sklearn</span>.<span class="pl-s1">ensemble</span>.<span class="pl-v">RandomForestClassifier</span>())))</pre></div>
<p dir="auto">I could not find a related issue and am willing to work on this if this is considered worth adding to scikit-learn.</p>