Research on MapReduce Parallel Optimization Method Based on Improved K means Clustering Algorithm
The traditional K means clustering algorithm occupies a large quantity of memory resources and computing costs when dealing with massive data. It is easy to be restricted by something such as the initial center point as well abnormal data, and usually can not achieve effective clustering of large scale data. In order to effectively solve the limitations of the algorithm, we propose a MapReduce parallel optimization method based on improved K means clustering algorithm. Firstly, differential evolution theory is introduced to determine the optimal initial clustering center, after that, on the basis of the influence of samples on clustering results, the corresponding weighted Euclidean distance is designed to achieve effective data differentiation, so as to effectively reduce the impact of samples on clustering analysis. The negative effect of abnormal data on clustering analysis can improve the accuracy of clustering. Finally, MapReduce programming model is used to realize parallel clustering. We use UCI datasets to verify the parallel optimization method. From the experimental results we can clearly know that the method we proposed has relatively stable parallel clustering results, faster operation speed, and effectively saves the operation time.
DSIT