How to deal with visually overlapping clusters I have done hierarchal clustering (average) and silhouette method shows 6 as the best k. However visualising I see that most of the clusters are overlapping. Any way to deal with the overlapping clusters? Does overlapping clusters show the problem with clustering?
Some stats: $average.between
0.8199681
$average.within
0.5613382
$n.between
18215
$n.within
5438
$max.diameter
1 1
$min.separation
0.1666667
$within.cluster.ss
37.59716
$clus.avg.silwidths
(1)0.2679033  (2)0.2181911 (3)0.1480349 (4)0.1988496 (5)0.3415294 (6)0.2011643
$avg.silwidth
0.2187254


hopkins(Cluster, n=nrow(Cluster)-1) ## Seems to be clusterable as the value is 0.36 
## Also checked VAT
library(factoextra)
fviz_dist(dist.mat, show_labels = FALSE, order = TRUE)+ labs(title = "SR")
##Dist mat
library(vegan)
dist.mat<-vegdist(Cluster, method="jaccard", binary = TRUE)
##Cluster
Cl.hclust <- hclust(dist.mat, method="average")
Cl.cuts <- cutree(Cl.hclust, k=6)
# Statistics for clustering
library(fpc)
hc_stats <- cluster.stats(dist.mat,  Cl.cuts) # indices
hc_stats
##Visualize 
fviz_cluster(list(data = Clusttr2, cluster = Cl.cuts), geom="point") + ggtitle("Agglomerative clustering (average)")   ```



I think the overlap is a visual artifact of reducing a high-dimensional space to a 2D space. I guess you used some PCA or multidimensional scaling technique to produce your plot? If I interpret your figure correctly, your "Dim1" captures only 15.9% of the variation so I would say that two dimensions are not very good at representing the whole dataset.
After cutting a tree produced by hierarchical clustering, a data point should belong to only one cluster. Perhaps you should be concerned about whether 6 clusters are not too many, but that depends on what you want to do with the clusters and how much separation you can reasonably expect.
