Answers.Body: <p>In the paper <a href="http://aboumoustafa.org/KAM_ECML2015.pdf" rel="nofollow noreferrer">Generalization in Unsupervised Learning</a> (2015), <a href="http://aboumoustafa.org/" rel="nofollow noreferrer">Abou-Moustafa</a> and Schuurmans develop an approach to assess the generalization of an unsupervised learning algorithm <span class="math-container">$A$</span> on a given dataset <span class="math-container">$S$</span> and how to compare the generalization ability of two unsupervised learning algorithms <span class="math-container">$A_1$</span> and <span class="math-container">$A_2$</span>, for the same learning task.</p>
  
  <p>They first provide a more abstract and general definition of an unsupervised learning algorithm and loss function. Then they define the expected risk, empirical risk and generalization gap in a similar way to the case of supervised learning. Finally, they derive an upper bound on <span class="math-container">$A$</span>'s expected loss.</p>
  
  <p>Of course, you should read the paper for more details. Specifically, section 2 (page 3) describes their setting in detail.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-19T23:14:11.527Z
Answers.LastActivityDate: 2020-04-19T23:31:09.580Z
Answers.LastEditDate: 2020-04-19T23:31:09.580Z
Answers.UserDisplayName: nbro
Body: <p>I've been learning a little bit about generalization theory, and in particular, the PAC (and PAC-Bayes) approach to thinking about this problem.</p>
  
  <p>So, I started to wonder if there is an analogous version of "generalization" in Unsupervised Learning? I.e., is there a general framework that encapsulates how "good" an unsupervised learning method is? There's reconstruction error for learning lower dimensional representations, but what about unsupervised clustering?</p>
  
  <p>Any ideas?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-19T23:08:03.143Z
Comments.Text: Cross-posted: https://ai.stackexchange.com/q/20382/1794, https://cstheory.stackexchange.com/q/46660/5038.  Please [do not post the same question on multiple sites](https://meta.stackexchange.com/q/64068).
Comments.UserDisplayName: D.W.
Tags: <machine-learning><unsupervised-learning><generalization><pac-learning>
Title: Is there a notion of generalization in unsupervised learning?
UserDisplayName: Marcel
ThreadText: # Is there a notion of generalization in unsupervised learning?
  
  (Asked by: username_0 on 2020-04-19)
  
  I've been learning a little bit about generalization theory, and in particular, the PAC (and PAC-Bayes) approach to thinking about this problem.
  
  So, I started to wonder if there is an analogous version of "generalization" in Unsupervised Learning? I.e., is there a general framework that encapsulates how "good" an unsupervised learning method is? There's reconstruction error for learning lower dimensional representations, but what about unsupervised clustering?
  
  Any ideas?
  
  **Tags:** machine-learning, unsupervised-learning, generalization, pac-learning
  
  > **Comments:**
  > **username_1**: Cross-posted: https://ai.stackexchange.com/q/20382/1794, https://cstheory.stackexchange.com/q/46660/5038.  Please [do not post the same question on multiple sites](https://meta.stackexchange.com/q/64068).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-04-19. Score: 1 | Accepted answer
  
  In the paper [Generalization in Unsupervised Learning](http://aboumoustafa.org/KAM_ECML2015.pdf) (2015), [Abou-Moustafa](http://aboumoustafa.org/) and Schuurmans develop an approach to assess the generalization of an unsupervised learning algorithm $A$ on a given dataset $S$ and how to compare the generalization ability of two unsupervised learning algorithms $A\_1$ and $A\_2$, for the same learning task.
  
  They first provide a more abstract and general definition of an unsupervised learning algorithm and loss function. Then they define the expected risk, empirical risk and generalization gap in a similar way to the case of supervised learning. Finally, they derive an upper bound on $A$'s expected loss.
  
  Of course, you should read the paper for more details. Specifically, section 2 (page 3) describes their setting in detail.
  
  
  ---
  
  