Answers.Body: <p>Its not required, you can have <span class="math-container">$m=1$</span>, actually it can be any number <span class="math-container">$\geq 1$</span>.   </p>
  
  <p>Now the better question is why to have it? The answer is that it adds a smoothing effect. Lets look at it in each of the limits (<span class="math-container">$\lim m \rightarrow 1$</span> and <span class="math-container">$\lim m \rightarrow \infty$</span>)  </p>
  
  <p>Towards <span class="math-container">$\infty$</span>, it makes <span class="math-container">$u_{ij}$</span> equal to <span class="math-container">$\frac{1}{c}$</span>, making each point have equal membership of each class regardless of <span class="math-container">$m$</span>. From the optimization perspective, its saying how can we achieve finding clusters that are closest to all points, therefore by definition it has already achieved that, and so the Loss will always be 0. (at its global minimum)  </p>
  
  <p>Now in the other limit, the constants are inversely proportional to the square of the normalized euclidean distance. This makes intuitive sense, the membership is high if they are close, and the membership is low if they are not (relatively)  </p>
  
  <p>So why do we have the <span class="math-container">$m$</span>, its for control. It allows us to choose and experiment with how heavy each distance should hold weight in the membership. An example where a larger <span class="math-container">$m$</span> may be useful is when the data isnt clean, and you dont want to rely so heavily on euclidean distance as the membership, so you forcibly add in a smoothing effect </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-23T19:46:45.723Z
Answers.Text: Assume I have two clusters, then for a point $x_1$ the $u_{ij}$ must sum to $1$. So I could have for example $u_{11}=0.8$ and $u_{12}=0.2$. If I now let $m\rightarrow\infty$, then $0.8^\infty=0$ and $0.2^\infty=0$, not $\frac{1}{2}$. How did you come up with the $\frac{1}{C}$?
Answers.UserDisplayName: user9007131
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-23T20:11:45.493Z
Answers.Text: The u’s are a function of m as well
Answers.UserDisplayName: mshlis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-24T17:29:09.153Z
Answers.Text: I don't understand. It's still $0.8^m \rightarrow 0$ and not $0.8^m\rightarrow\frac{1}{2}$ for $m\rightarrow\infty$
Answers.UserDisplayName: user9007131
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-24T17:30:54.417Z
Answers.Text: the value is still 0, the membership though is $\frac{1}{2}$... If $m \rightarrow \infty$ the problem is meaningless because the membership is no longer a function of any of the cetroids but is gauranteed to be equally associated with each
Answers.UserDisplayName: mshlis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-24T17:32:49.050Z
Answers.Text: i adjusted my answer slightly to reflect that
Answers.UserDisplayName: mshlis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-23T19:35:16.033Z
Answers.LastActivityDate: 2019-07-24T17:32:07.377Z
Answers.LastEditDate: 2019-07-24T17:32:07.377Z
Answers.UserDisplayName: mshlis
Body: <p>According to my lecture, Fuzzy c-Means tries to minimize the following objective function:</p>
  
  <p><span class="math-container">$$J(X,B,U)=\sum_{i=1}^c\sum_{j=1}^n u_{ij}^w \, d^2(\vec{\beta_i},\vec{x_j})$$</span></p>
  
  <p>where <span class="math-container">$X$</span> are the data points, <span class="math-container">$B$</span> are the cluster-'prototypes', and <span class="math-container">$U$</span> is the matrix containing the fuzzy membership degrees. <span class="math-container">$d$</span> is a distance measure.</p>
  
  <p>A constraint is that the membership degrees for a single datapoint w.r.t. all clusters sum to <span class="math-container">$1$</span>: <span class="math-container">$\sum_{j=1}^n\, u_{ij}=1$</span>.</p>
  
  <p>Now in the first equation, what is the role of the <span class="math-container">$w$</span>? I read that one could use any convex function instead of <span class="math-container">$(\cdot)^w$</span>. But why use anything at all. Why don't we just use the membership degrees? My lecture says using the fuzzifier is necessary but doesn't explain why.</p>
  
Tags: <unsupervised-learning><fuzzy-logic><k-means><clustering>
Title: What is the role of the 'fuzzifier' w in Fuzzy Clustering?
UserDisplayName: user9007131
ThreadText: # What is the role of the 'fuzzifier' w in Fuzzy Clustering?
  
  (Asked by: username_0 on 2019-07-23)
  
  According to my lecture, Fuzzy c-Means tries to minimize the following objective function:
  
  $$J(X,B,U)=\sum\_{i=1}^c\sum\_{j=1}^n u\_{ij}^w \, d^2(\vec{\beta\_i},\vec{x\_j})$$
  
  where $X$ are the data points, $B$ are the cluster-'prototypes', and $U$ is the matrix containing the fuzzy membership degrees. $d$ is a distance measure.
  
  A constraint is that the membership degrees for a single datapoint w.r.t. all clusters sum to $1$: $\sum\_{j=1}^n\, u\_{ij}=1$.
  
  Now in the first equation, what is the role of the $w$? I read that one could use any convex function instead of $(\cdot)^w$. But why use anything at all. Why don't we just use the membership degrees? My lecture says using the fuzzifier is necessary but doesn't explain why.
  
  **Tags:** unsupervised-learning, fuzzy-logic, k-means, clustering
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-07-23. Score: 1
  
  Its not required, you can have $m=1$, actually it can be any number $\geq 1$.
  
  Now the better question is why to have it? The answer is that it adds a smoothing effect. Lets look at it in each of the limits ($\lim m \rightarrow 1$ and $\lim m \rightarrow \infty$)
  
  Towards $\infty$, it makes $u\_{ij}$ equal to $\frac{1}{c}$, making each point have equal membership of each class regardless of $m$. From the optimization perspective, its saying how can we achieve finding clusters that are closest to all points, therefore by definition it has already achieved that, and so the Loss will always be 0. (at its global minimum)
  
  Now in the other limit, the constants are inversely proportional to the square of the normalized euclidean distance. This makes intuitive sense, the membership is high if they are close, and the membership is low if they are not (relatively)
  
  So why do we have the $m$, its for control. It allows us to choose and experiment with how heavy each distance should hold weight in the membership. An example where a larger $m$ may be useful is when the data isnt clean, and you dont want to rely so heavily on euclidean distance as the membership, so you forcibly add in a smoothing effect
  
  > **Comments:**
  > **username_0**: Assume I have two clusters, then for a point $x_1$ the $u_{ij}$ must sum to $1$. So I could have for example $u_{11}=0.8$ and $u_{12}=0.2$. If I now let $m\rightarrow\infty$, then $0.8^\infty=0$ and $0.2^\infty=0$, not $\frac{1}{2}$. How did you come up with the $\frac{1}{C}$?  
  > **username_1**: The u’s are a function of m as well  
  > **username_0**: I don't understand. It's still $0.8^m \rightarrow 0$ and not $0.8^m\rightarrow\frac{1}{2}$ for $m\rightarrow\infty$  
  > **username_1**: the value is still 0, the membership though is $\frac{1}{2}$... If $m \rightarrow \infty$ the problem is meaningless because the membership is no longer a function of any of the cetroids but is gauranteed to be equally associated with each  
  > **username_1**: i adjusted my answer slightly to reflect that  
  
  ---
  
  