How did you calculate the sparsity loss?
Based on your description it seems as if you’ve implemented a “counting” approach, which might not be differentiable.
To verify it, you could check, if the sparsity loss has a valid .grad_fn.
If that’s not the case, your approach is currently just adding an offset to the loss, which won’t have an effect on the training.
In case your sparsity loss doesn’t have a .grad_fn, you could maybe try to add an L1 penalty to the parameters of the last layer of your encoder, which might make them sparse.