Body: <p>My autoencoder give latent vectors with many zeroes components like:</p>
  <pre><code>[3.0796502  2.9488854  0.9002177  0.         0.         0.
   0.         0.         0.         1.0181859  0.         0.68507403
   0.         0.6128702  0.         0.         0.         0.
   0.         1.763725   0.         0.         0.         1.0947669
   0.         1.5330162  0.         0.         0.         0.
   0.         1.7434856  0.         0.         1.8942142  2.0379465
   0.         0.         1.2500542  0.         0.         0.
   0.         0.         0.         2.7917862  0.         0.
   2.2105153  0.         0.         0.         1.5798858  0.
   0.         3.7405093  0.8692952  0.01490922 0.         0.
   2.8320081  0.         0.         0.        ]
  </code></pre>
  <p>Certain components are always zero, another not always. Why might this be happening? How can I figure this out?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-31T15:58:09.500Z
Comments.Text: Ok, there was ReLu() activation on last layer of decoder.
Comments.UserDisplayName: Dmitry  Sokolov
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-31T16:07:04.573Z
Comments.Text: Hello. Welcome to AI SE. If you solved your problem or you have the answer to your question, feel free to write a formal answer below ;)
Comments.UserDisplayName: nbro
Tags: <neural-networks><autoencoders>
Title: Why would an auto-encoder produce latent vectors with many zeros?
UserDisplayName: Dmitry  Sokolov
ThreadText: # Why would an auto-encoder produce latent vectors with many zeros?
  
  (Asked by: username_0 on 2021-05-31)
  
  My autoencoder give latent vectors with many zeroes components like:
  [3.0796502  2.9488854  0.9002177  0.         0.         0.
   0.         0.         0.         1.0181859  0.         0.68507403
   0.         0.6128702  0.         0.         0.         0.
   0.         1.763725   0.         0.         0.         1.0947669
   0.         1.5330162  0.         0.         0.         0.
   0.         1.7434856  0.         0.         1.8942142  2.0379465
   0.         0.         1.2500542  0.         0.         0.
   0.         0.         0.         2.7917862  0.         0.
   2.2105153  0.         0.         0.         1.5798858  0.
   0.         3.7405093  0.8692952  0.01490922 0.         0.
   2.8320081  0.         0.         0.        ]
  
  Certain components are always zero, another not always. Why might this be happening? How can I figure this out?
  
  **Tags:** neural-networks, autoencoders
  > **Comments:**
  > **username_0**: Ok, there was ReLu() activation on last layer of decoder.  
  > **username_1**: Hello. Welcome to AI SE. If you solved your problem or you have the answer to your question, feel free to write a formal answer below ;)  
  
  ---
  
  ## 0 answers
  
  