Hi, I know that his question is silly, but I have a big hole in my understanding.
KL loss helps me to make input dist similar to target dist. , I recently see it more and more in paper, where I have some FeatureMap tensor, and they want this tensor to look like another tensor(target) (doing transfer learning, knowledge distillation,so on), and I wonder does, KL loss is able to get an input tensor or I need to do some “categorical sampling” with Gumbel softmax tricks?
Maybe the question makes no sense, but I have to know this important detail.
Thanks!