Parallel Implementation of Statistical DBSCAN Algorithm for Spark-based Clustering on Google Cloud Platform

Abstract

We present a new parallel DBSCAN algorithm for Spark on the Google Cloud Platform. Statistical analysis is applied to determine DBSCAN's optimal parameters to enhance clustering performance. for scalability Cost-based, R-tree partitioning is selected based on the distribution of the dataset into balanced workloads. Parallel DBSCAN consists of three parts: local DBSCAN, partitioning, and merging. Optimising partitioning of parallel DBSCAN is important to save time and space compared to serial DBSCAN. This approach can improve the performance of large datasets. First, the modified DBSCAN is applied to the UCI standard datasets. Basic benchmark clustering Second, using the COVID-19 dataset by the Johns Hopkins University Center for Systems Science and Engineering, we present a temporal analysis of the number of new cases and deaths among countries using artificial intelligence. For big data, countries' clustering is determined, and we study a special case of Egypt