Is using a Dense layer with a lower feature-space, a legitimate way of dimensionality-reduction ?
Also are PCA and t-SNE good options for dimensionality-reduction of embeddings computed from transformer-based models? I see that they are performed a lot with Word2Vec/ TF.IDF embeddings