Award-winning Work Information & Description
Visualization Analysis of Lanzhou House Price Based on Selenium Automation Web Crawler
Description
This work is mainly to use the selenium framework to automatically crawl the chain home online Lanzhou city housing rental and sales forecast analysis, in order to understand the current city's new housing prices, from the perspective of housing prices to explore the most livable areas for this analysis. Under the positioning of "housing, not speculation", housing purchase has become a more concerned issue for the general public. The big data analysis technology is introduced into the analysis of house prices, and the Scrapy crawler framework is used to crawl the online data of Lanzhou house prices. After cleaning and visualization, the factors affecting house prices are presented in a visual form. Compared with traditional methods, big data analysis technology has obvious advantages in data collection and visualization applications. The work mainly has five parts, namely data acquisition, data preprocessing, data storage, data analysis, data visualization. The first part is data collection, execution program crawling, the selected data is the data of the chain home network, focusing on crawling the data of Lanzhou city, the main function is to use the selenium framework to automatically crawl the data of the chain home network Lanzhou city housing. The second part is data preprocessing, which is to process the raw data to get higher quality data. Remove room records with an area greater than 10000 square meters and delete all empty columns. According to the type of furniture, the location and area of the house, the house price is classified and summarized, and the house price is divided into sections. The third part is data storage, which is stored in the database. The fourth part is data analysis, data integration analysis, using SQL statement update, each time the program will update the data, so that the data can achieve real-time dynamic effect. Part 5 is data visualization, the core of which is to convert boring data into a chart format to make the data more intuitive. After investigation, it is found that Chengguan District is the main commercial area of Lanzhou, with less industrial pollution sources, mainly from traffic pollution, but Chengguan District is the first urban area to develop in Lanzhou, and it is also the political, economic and cultural center of Lanzhou. Supporting resources and urban functions are also more abundant, so there are more people renting and buying houses in Chengguan District.