Answers.Body: <p>I did an experiment, took a trained densenet121 and kept the bottom layers. I trained the FC layer to a softmax and then to a lambda layer that normalized the vector. I trained the network with imagenet to make the outputs the most far a away from (1,1,1,1,1...1) as possible, so I would get one hot vectors. I did, but the network trained to a single category (put all in the same hot vector). Then I added a penalty that encourages to make vectors different is their features are also different. it improved a little, a dozen categories instead of one, bout noting close to a thousand, the available number.</p>
  
  <p>I am posting this so nobody wastes his time in a silly idea like this one.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-30T00:41:43.780Z
Answers.LastActivityDate: 2019-11-30T00:50:11.103Z
Answers.LastEditDate: 2019-11-30T00:50:11.103Z
Answers.UserDisplayName: Wolphram jonny
Body: <p>When training a deep network to learn object classification from a set like ImageNet, we minimize the cross entropy between the ground truth and the predicted  categories. This is done in a supervised way. It is my understanding that you can separate categories in an unsupervised way using principal component analysis, but I have never seen that in a deep network. I am curious if this can be done easily in the last case. One possible way to do this would be to minimize a loss that favors categorization into one-hot vectors (this would only guarantee that an image is classified into a single category, rather that the correct category, though). Has this been done, or is there any reason why not?</p>
  
Tags: <deep-learning><object-recognition><unsupervised-learning>
Title: Are there methods that allow deep networks to learn object categorization in a self-supervised way?
UserDisplayName: Wolphram jonny
ThreadText: # Are there methods that allow deep networks to learn object categorization in a self-supervised way?
  
  (Asked by: username_0 on 2019-10-12)
  
  When training a deep network to learn object classification from a set like ImageNet, we minimize the cross entropy between the ground truth and the predicted categories. This is done in a supervised way. It is my understanding that you can separate categories in an unsupervised way using principal component analysis, but I have never seen that in a deep network. I am curious if this can be done easily in the last case. One possible way to do this would be to minimize a loss that favors categorization into one-hot vectors (this would only guarantee that an image is classified into a single category, rather that the correct category, though). Has this been done, or is there any reason why not?
  
  **Tags:** deep-learning, object-recognition, unsupervised-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2019-11-30. Score: 1 | Accepted answer
  
  I did an experiment, took a trained densenet121 and kept the bottom layers. I trained the FC layer to a softmax and then to a lambda layer that normalized the vector. I trained the network with imagenet to make the outputs the most far a away from (1,1,1,1,1...1) as possible, so I would get one hot vectors. I did, but the network trained to a single category (put all in the same hot vector). Then I added a penalty that encourages to make vectors different is their features are also different. it improved a little, a dozen categories instead of one, bout noting close to a thousand, the available number.
  
  I am posting this so nobody wastes his time in a silly idea like this one.
  
  
  ---
  
  