I’m not sure what exactly is causing this behavior, but you might want to play around with hyperparameters in TSNE, as it could show different outputs based on its setup.
E.g. the paper and conclusions mentioned in this tweet might be interesting for you to check out.