Award-winning Work Information & Description
Intelligent wetland waterfowl monitoring system based on multi-modal recognition technology
Description
How to deal with the collected images and sounds of wetlands and waterbirds and other information, and to extract and classify the information has become the main problem to be studied. Based on the extracted audio and video information, using the multi-modal features to identify waterbirds is the second problem to be solved in this project. The project is mainly divided into three main modules, the first module is about the identification of waterbird populations, the module through the user to upload images, video or audio, identify the species of waterbirds, the number of waterbirds contained in the population and waterbird behavior. At present, the traditional detection method is still used for the observation of waterbirds in wetland nature reserves, which mainly depends on the manual visual interpretation of experience, which not only has higher requirements for the work experience of monitoring personnel, but also puts forward higher requirements for real-time waterbird monitoring with the increase of data acquisition means. Therefore, the main research problem of this module is how to integrate the collected image, video and audio data into multi-modal. The use of biometric fusion technology from multiple dimensions, the use of a variety of means, based on multiple platforms to improve the performance of the identification system, reduce noise, reduce intra-class, inter-class differences, improve the accuracy of waterbird recognition. It can also identify the population size, quantity and age of waterbirds more accurately. The application of multi-modal technology in the system can also improve the efficiency of real-time comparison between images and targets. For example, in the process of comparing the characteristics of various behaviors of waterbirds, the length of time to complete a recognition will be unimaginable. Therefore, the accuracy and efficiency of recognition can be improved by performing multi-modal fusion of data. The second module of the project is about the identification of rare waterbirds. Because they are relatively rare waterbirds, the sample data of the waterbirds are relatively small and lack of data labeling. Therefore, this module mainly studies how to solve the problem of insufficient training samples of waterbirds. The third module of the project is the dynamic monitoring of wetland area. The problem studied in this module is how to obtain clear remote sensing images of wetlands and extract wetland images from remote sensing images. Finally, based on the above modules, the project team uses the Flask framework to integrate the three main modules into the final application. The main functions of the project include: in the aspect of waterbird identification: 1. Upload bird photos, identify the species of birds, and comprehensively analyze the approximate age of the waterbird according to the body shape and posture of the uploaded waterbird, which is divided into juvenile, adult and old, and the meaning represented by the modified waterbird type, so as to facilitate the monitoring of the personnel in the reserve. 2. According to the photos of the uploaded waterbird population, identify the number of waterbirds and analyze the size of the population. 3. Upload the sound of waterbirds to identify the species of waterbirds; in terms of wetland monitoring: 1. Calculate the current wetland area according to the uploaded pictures and the proportion of the pictures to the actual wetland area. 2. According to the uploaded video, the video is edited into a single frame picture for analysis, so as to realize the real-time monitoring of the regional wetland area periodically. Finally, the application can feed back the results of waterbird identification to waterbird enthusiasts, as well as data on wetland trends and information about waterbird populations to wetland conservation and management organizations.