<p dir="auto">I know that there are several previously created issues regarding this. However, I still don't understand why warm_start is not supported for liblinear solver? Whether this feature is going to be added in future? Based on my experience (of liblinear in C++), using warm_start can speedup training 2-4x times.</p>
<p dir="auto">NCA checks its parameters to have a specific type, which can easily fail in a GridSearch due to how param grid is made.</p>
<p dir="auto">Here is an example:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import numpy as np

from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
from sklearn.neighbors import NeighborhoodComponentsAnalysis
from sklearn.neighbors import KNeighborsClassifier

X = np.random.random_sample((100, 10))
y = np.random.randint(2, size=100)

nca = NeighborhoodComponentsAnalysis()
knn = KNeighborsClassifier()

pipe = Pipeline([('nca', nca),
                 ('knn', knn)])
                
params = {'nca__tol': [0.1, 0.5, 1],
          'nca__n_components': np.arange(1, 10)}
          
gs = GridSearchCV(estimator=pipe, param_grid=params, error_score='raise')
gs.fit(X,y)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">pipeline</span> <span class="pl-k">import</span> <span class="pl-v">Pipeline</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">model_selection</span> <span class="pl-k">import</span> <span class="pl-v">GridSearchCV</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">neighbors</span> <span class="pl-k">import</span> <span class="pl-v">NeighborhoodComponentsAnalysis</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">neighbors</span> <span class="pl-k">import</span> <span class="pl-v">KNeighborsClassifier</span>

<span class="pl-v">X</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">random_sample</span>((<span class="pl-c1">100</span>, <span class="pl-c1">10</span>))
<span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">randint</span>(<span class="pl-c1">2</span>, <span class="pl-s1">size</span><span class="pl-c1">=</span><span class="pl-c1">100</span>)

<span class="pl-s1">nca</span> <span class="pl-c1">=</span> <span class="pl-v">NeighborhoodComponentsAnalysis</span>()
<span class="pl-s1">knn</span> <span class="pl-c1">=</span> <span class="pl-v">KNeighborsClassifier</span>()

<span class="pl-s1">pipe</span> <span class="pl-c1">=</span> <span class="pl-v">Pipeline</span>([(<span class="pl-s">'nca'</span>, <span class="pl-s1">nca</span>),
                 (<span class="pl-s">'knn'</span>, <span class="pl-s1">knn</span>)])
                
<span class="pl-s1">params</span> <span class="pl-c1">=</span> {<span class="pl-s">'nca__tol'</span>: [<span class="pl-c1">0.1</span>, <span class="pl-c1">0.5</span>, <span class="pl-c1">1</span>],
          <span class="pl-s">'nca__n_components'</span>: <span class="pl-s1">np</span>.<span class="pl-en">arange</span>(<span class="pl-c1">1</span>, <span class="pl-c1">10</span>)}
          
<span class="pl-s1">gs</span> <span class="pl-c1">=</span> <span class="pl-v">GridSearchCV</span>(<span class="pl-s1">estimator</span><span class="pl-c1">=</span><span class="pl-s1">pipe</span>, <span class="pl-s1">param_grid</span><span class="pl-c1">=</span><span class="pl-s1">params</span>, <span class="pl-s1">error_score</span><span class="pl-c1">=</span><span class="pl-s">'raise'</span>)
<span class="pl-s1">gs</span>.<span class="pl-en">fit</span>(<span class="pl-v">X</span>,<span class="pl-s1">y</span>)</pre></div>
<p dir="auto">The issue is that for <code class="notranslate">tol</code>: 1 is not a float, and for  <code class="notranslate">n_components</code>: np.int64 is not int</p>
<p dir="auto">Before proposing a fix for this specific situation, I'd like to have your general opinion about parameter checking.<br>
I like this idea of common parameter checking tool introduced with the NCA PR. What do you think about extending it across the code-base (or at least for new or recent estimators) ?</p>
<p dir="auto">Currently parameter checking is not always done or often partially done, and is quite redundant. For instance, here is the input validation of lda:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="def _check_params(self):
        &quot;&quot;&quot;Check model parameters.&quot;&quot;&quot;
        if self.n_components &lt;= 0:
            raise ValueError(&quot;Invalid 'n_components' parameter: %r&quot;
                             % self.n_components)

        if self.total_samples &lt;= 0:
            raise ValueError(&quot;Invalid 'total_samples' parameter: %r&quot;
                             % self.total_samples)

        if self.learning_offset &lt; 0:
            raise ValueError(&quot;Invalid 'learning_offset' parameter: %r&quot;
                             % self.learning_offset)

        if self.learning_method not in (&quot;batch&quot;, &quot;online&quot;):
            raise ValueError(&quot;Invalid 'learning_method' parameter: %r&quot;
                             % self.learning_method)"><pre class="notranslate"><span class="pl-k">def</span> <span class="pl-en">_check_params</span>(<span class="pl-s1">self</span>):
        <span class="pl-s">"""Check model parameters."""</span>
        <span class="pl-k">if</span> <span class="pl-s1">self</span>.<span class="pl-s1">n_components</span> <span class="pl-c1">&lt;=</span> <span class="pl-c1">0</span>:
            <span class="pl-k">raise</span> <span class="pl-v">ValueError</span>(<span class="pl-s">"Invalid 'n_components' parameter: %r"</span>
                             <span class="pl-c1">%</span> <span class="pl-s1">self</span>.<span class="pl-s1">n_components</span>)

        <span class="pl-k">if</span> <span class="pl-s1">self</span>.<span class="pl-s1">total_samples</span> <span class="pl-c1">&lt;=</span> <span class="pl-c1">0</span>:
            <span class="pl-k">raise</span> <span class="pl-v">ValueError</span>(<span class="pl-s">"Invalid 'total_samples' parameter: %r"</span>
                             <span class="pl-c1">%</span> <span class="pl-s1">self</span>.<span class="pl-s1">total_samples</span>)

        <span class="pl-k">if</span> <span class="pl-s1">self</span>.<span class="pl-s1">learning_offset</span> <span class="pl-c1">&lt;</span> <span class="pl-c1">0</span>:
            <span class="pl-k">raise</span> <span class="pl-v">ValueError</span>(<span class="pl-s">"Invalid 'learning_offset' parameter: %r"</span>
                             <span class="pl-c1">%</span> <span class="pl-s1">self</span>.<span class="pl-s1">learning_offset</span>)

        <span class="pl-k">if</span> <span class="pl-s1">self</span>.<span class="pl-s1">learning_method</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> (<span class="pl-s">"batch"</span>, <span class="pl-s">"online"</span>):
            <span class="pl-k">raise</span> <span class="pl-v">ValueError</span>(<span class="pl-s">"Invalid 'learning_method' parameter: %r"</span>
                             <span class="pl-c1">%</span> <span class="pl-s1">self</span>.<span class="pl-s1">learning_method</span>)</pre></div>
<p dir="auto">most params aren't checked and for those who are there's a lot of duplicated code.</p>
<p dir="auto">A propose to be upgrade the new tool to be able to check open/closed intervals (currently only closed) and list membership.</p>
<p dir="auto">The api would be something like that:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="check_param(param, name, valid_options)"><pre class="notranslate"><code class="notranslate">check_param(param, name, valid_options)
</code></pre></div>
<p dir="auto">where valid_options would be a dict of <code class="notranslate">type: constraint</code>. e.g for the <code class="notranslate">beta_loss</code> param of <code class="notranslate">NMF</code>, it can be either a float or a string in a list, which would give</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="valid_options = {numbers.Real: None,  # None for no constraint
                 str: ['frobenius', 'kullback-leibler', 'itakura-saito']}"><pre class="notranslate"><code class="notranslate">valid_options = {numbers.Real: None,  # None for no constraint
                 str: ['frobenius', 'kullback-leibler', 'itakura-saito']}
</code></pre></div>
<p dir="auto">Sometimes a parameter can only be positive or within a given interval, e.g. <code class="notranslate">l1_ratio</code> of <code class="notranslate">LogisticRegression</code> must be between 0 and 1, which would give</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="valid_options = {numbers.Real: Interval(0, 1, closed='both')}"><pre class="notranslate"><code class="notranslate">valid_options = {numbers.Real: Interval(0, 1, closed='both')}
</code></pre></div>
<p dir="auto">positivity of e.g. <code class="notranslate">max_iter</code> would be <code class="notranslate">numbers.Integral: Interval(left=1)</code>.</p>