Can you try to count the number of non-zero entries in the feature output?
I have noticed the same sometime ago in VGGFace. The features are really sparse and this would be due to the ReLU unit. Nevertheless, the features (from VGGFace) are pretty good to use in Face recognition atleast.
I think, you need not worry about the nature of features unless they perform really poor in the task.