Dimension Reduction and Clustering on Big Dataset I am currently working on a marketing analytics project and my dataset is rather large: 400K rows and 70 features which contains both continuous and binary variables.
I've tried using PCA and incorporating the results into a Kmeans clustering algorithm but


*

*There doesn't appear to be a clear optimal number of clusters when I utilize the elbow method and processing time for silhouette scores analysis is really long
2.I've seen that computing a gowers matrix is best for datasets with mixed datatypes, however I do not have enough storage using Python to compute a 400Kx400K matrix.


Are there any suggestions? Perhaps there is a sampling method, but I am not sure how to then apply the clusters created in the sample data to the rest of the dataset. Any help is appreciated!!!

I think you could choose n samples randomly(and taking care of how well they could symbol the whole dataset). After getting the clusters, you could classify the remaining samples to clusters based on their distances to different clusters.
