<p dir="auto">Original title: <em><strong>"Cosine" affinity type in FeatureAgglomeration somehow casue memory overflow in a particular dataset</strong></em></p>


<h4 dir="auto">Description</h4>

<p dir="auto">Please carefully test the codes below! Using "cosine" affinity type in FeatureAgglomeration, the codes will cause memory overflow with a particular dataset (<a href="http://gleason.case.edu/webdata/tpot/" rel="nofollow">Download here</a>). It is all right with other affinity types. And this issue cannot be reproduced using simulation data (like using make_classification in sci-kit learn). Not sure why it happened.</p>
<h4 dir="auto">Steps/Code to Reproduce</h4>

<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from sklearn.cluster import FeatureAgglomeration
import numpy as np
import time
train_data = np.genfromtxt('fold_2_trainFeatVec.csv', delimiter=',')
train_labels= np.genfromtxt('fold_2_trainLabels.csv', delimiter=',')


fa = FeatureAgglomeration(affinity=&quot;cosine&quot;, linkage=&quot;average&quot;) # no matter linage =&quot;average&quot; or &quot;complete&quot;
time_start= time.time()
fa.fit(train_data, train_labels) # memory keeps increasing here
time_end = time.time()
print('Time usage:',time_end-time_start)
"><pre class="notranslate"><code class="notranslate">from sklearn.cluster import FeatureAgglomeration
import numpy as np
import time
train_data = np.genfromtxt('fold_2_trainFeatVec.csv', delimiter=',')
train_labels= np.genfromtxt('fold_2_trainLabels.csv', delimiter=',')


fa = FeatureAgglomeration(affinity="cosine", linkage="average") # no matter linage ="average" or "complete"
time_start= time.time()
fa.fit(train_data, train_labels) # memory keeps increasing here
time_end = time.time()
print('Time usage:',time_end-time_start)

</code></pre></div>
<p dir="auto">Via <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="520559841" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/15583" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/15583/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/15583">#15583</a>:<br>
it would be nice to have <code class="notranslate">sample_weight</code> support in <code class="notranslate">StandardScaler</code>. I don't see how to do that for sparse data right now, though (without touching the Cython code).</p>