<p dir="auto">I want to use <code class="notranslate">sklearn.compose.ColumnTransformer</code> consistently (not parallel, so, the second transformer should be executed only after the first) for intersecting lists of columns in this way:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="log_transformer = p.FunctionTransformer(lambda x: np.log(x))
df = pd.DataFrame({'a': [1,2, np.NaN, 4], 'b': [1,np.NaN, 3, 4], 'c': [1 ,2, 3, 4]})
compose.ColumnTransformer(n_jobs=1,
                         transformers=[
                             ('num', impute.SimpleImputer() , ['a', 'b']),
                             ('log', log_transformer, ['b', 'c']),
                             ('scale', p.StandardScaler(), ['a', 'b', 'c'])
                         ]).fit_transform(df)"><pre class="notranslate"><code class="notranslate">log_transformer = p.FunctionTransformer(lambda x: np.log(x))
df = pd.DataFrame({'a': [1,2, np.NaN, 4], 'b': [1,np.NaN, 3, 4], 'c': [1 ,2, 3, 4]})
compose.ColumnTransformer(n_jobs=1,
                         transformers=[
                             ('num', impute.SimpleImputer() , ['a', 'b']),
                             ('log', log_transformer, ['b', 'c']),
                             ('scale', p.StandardScaler(), ['a', 'b', 'c'])
                         ]).fit_transform(df)
</code></pre></div>
<p dir="auto">So, I want to use <code class="notranslate">SimpleImputer</code> for <code class="notranslate">'a'</code>, <code class="notranslate">'b'</code>, then <code class="notranslate">log</code> for <code class="notranslate">'b'</code>, <code class="notranslate">'c'</code>, and then <code class="notranslate">StandardScaler</code> for <code class="notranslate">'a'</code>, <code class="notranslate">'b'</code>, <code class="notranslate">'c'</code>.</p>
<p dir="auto">But:</p>
<ol dir="auto">
<li>I get array of <code class="notranslate">(4, 7)</code> shape.</li>
<li>I still get <code class="notranslate">Nan</code> in <code class="notranslate">a</code> and <code class="notranslate">b</code> columns.</li>
</ol>
<p dir="auto">So, how can I use <code class="notranslate">ColumnTransformer</code> for different columns in the manner of <code class="notranslate">Pipeline</code>?</p>
<p dir="auto">P.S. please note that I already <a href="https://stackoverflow.com/questions/62225230/consistent-columntransformer-for-intersecting-lists-of-columns" rel="nofollow">asked a question on SO.</a></p>
<p dir="auto"><a href="https://scikit-learn.org/stable/modules/clustering.html#k-means" rel="nofollow">In the documentation</a> , you wrote:</p>
<blockquote>
<p dir="auto">The algorithm supports sample weights, which can be given by a parameter sample_weight. This allows to assign more weight to some samples when computing cluster centers and values of inertia. For example, assigning a weight of 2 to a sample is equivalent to adding a duplicate of that sample to the dataset X.</p>
</blockquote>
<p dir="auto">This is disproved by the following code:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from sklearn.cluster import KMeans
vecs = [[0], [1]]
weights = [2 , 2]
kmeans = KMeans(n_clusters=1)
kmeans.fit(vecs, sample_weight=weights)
print(kmeans.inertia_) #Outputs 0.5

vecs = [[0], [0], [1], [1]]
weights = [1 , 1, 1, 1]
kmeans = KMeans(n_clusters=1)
kmeans.fit(vecs, sample_weight=weights)
print(kmeans.inertia_) #Outputs 1.0"><pre class="notranslate"><code class="notranslate">from sklearn.cluster import KMeans
vecs = [[0], [1]]
weights = [2 , 2]
kmeans = KMeans(n_clusters=1)
kmeans.fit(vecs, sample_weight=weights)
print(kmeans.inertia_) #Outputs 0.5

vecs = [[0], [0], [1], [1]]
weights = [1 , 1, 1, 1]
kmeans = KMeans(n_clusters=1)
kmeans.fit(vecs, sample_weight=weights)
print(kmeans.inertia_) #Outputs 1.0
</code></pre></div>
<p dir="auto">Suggested fix:<br>
Explain the correct definition of the inertia to <a href="https://scikit-learn.org/stable/modules/clustering.html#k-means" rel="nofollow">this site</a> and <a href="https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans" rel="nofollow">this site</a> . I would suggest adding the weights to the equation on <a href="https://scikit-learn.org/stable/modules/clustering.html#k-means" rel="nofollow">this site</a> .</p>