Abstract:
Big Data (BD) scraping systems are among the recommended approaches for large-scale web data extraction. However,
these systems for collecting large amounts of data face many challenges, including processing, storage, and data extraction
reliability. Due to its potentials, cloud computing is becoming a viable solution to support BD scraping systems.
This paper tenders a cloud based-web scraping framework for weather BD extraction and analysis. The aim is to extract weather BD
from web sources, analyze this data and use it for visualization and forecasting purposes, and this by enabling elastic and on-demand
resources. The framework is implemented using Selenium and Amazon Web Services and tested with Morocco weather data.
Analysis of the performance and scalability of the proposed cloud-based scrapper shows that it offers greater efficiency in terms of
data collection and analysis and in terms of forecast quality, thanks to the ability to take advantage of cloud resources.
